Methods and systems for providing relevant supplemental content to a user device

ABSTRACT

Systems and methods are provided for presenting a supplemental content asset on a secondary device that is relevant to a primary content asset that is being presented on a primary user device, to enhance the user experience of the primary content asset. A relevant supplemental content asset may be selected based on characteristics of the primary content asset and/or user profile information. Presentation of the supplemental content asset and primary content asset may be coordinated and synchronized by exchange of messages between the primary user device and secondary device.

BACKGROUND OF THE DISCLOSURE

This application relates to supplementing primary content assetsaccessed by a primary user device, by presentation of supplementalcontent assets on a secondary device.

Traditional systems allow users to access content assets for viewing andplay back from a single device or a single system. For example, atraditional home entertainment system may include television equipment,stereo equipment, set-top boxes and digital media disc players. The userequipment of the traditional home entertainment system are operatedindependently to access content assets, and do not interact.

Traditional systems do not enable user devices to supplement contentassets accessed by other devices. For example, in response to play backof a song on a portable media device, a traditional home entertainmentsystem is unable to play back a music video from its own content libraryrelated to the song.

SUMMARY OF THE DISCLOSURE

In view of the foregoing, methods and systems are provided for enablinginteraction between two different user devices to present a supplementalcontent asset. A primary content asset accessed by a primary user devicemay be supplemented by presenting a supplemental content asset on asecondary device. As an example, a system may include a television, aset-top box and a stereo. A user may tune the stereo to a radio stationand play a popular song on the stereo. In response to the playing of thesong, the television may supplement the popular song by displaying aslide show of images related to the popular song. The slide show may beprovided to the television from a set-top box that accessed the imagesfrom the Internet.

A user may operate a primary user device to access a primary contentasset. In response to the accessing of the primary content asset by theprimary user device, a secondary device may search for a supplementalcontent asset related to the primary content asset from a library offiles stored locally on the secondary device, or a library of filesstored remotely. The secondary device may select the most relevantcontent asset to supplement the primary content asset accessed by theprimary user device. Depending on usage, a user device may act as aprimary device or act as a secondary device. For example, when a firstuser device is used to access a primary content asset that is to besupplemented by a second user device, the first user device may be aprimary device, and the second user device may be a secondary device.Alternatively, when the first user device is used to provide asupplemental content asset to a primary content asset that is accessedon the second user device, the first user device may be the secondarydevice, and the second user device may be the primary device.

The supplementing of a primary content asset may be performed inresponse to detection by a secondary device, of a primary device withina proximity of the secondary device, or vice versa, in response todetection by a primary device, of a secondary device within a proximityof the primary device. A proximity may refer to a physical proximity. Asreferred to herein, a physical proximity may refer to a measurable rangewithin which detecting circuitry of an electronic device is able todetect a minimum signal. A measurable range may include a wireless rangewithin which a first wireless device is able to detect a wireless signalreceived from a second wireless device, a visual detection range withinwhich a biometric device such as a camera is able to resolve two points,an audio detection range within which a microphone is able to detectaudio, or any other suitable measurable range or any combinationthereof.

The presentation of a supplemental content asset on a secondary devicemay be coordinated with the presentation of a primary content asset on aprimary device, in response to a detection event. For example, inresponse to detecting a primary device, a secondary device may receiveplay back information about a primary content asset on the primarydevice. As referred to herein, “play back information” may include anelapsed play back time of the primary content asset, and statusinformation about whether the primary content asset is paused or playedback. In response to receiving the play back information, the secondarydevice may advance the start time of the supplemental content asset tocompensate for the elapsed play back time of the primary content asset.In this way, the presentation of the supplemental content asset canmatch the presentation of the primary content asset.

In some embodiments, the primary user device may detect a secondarydevice, and receive a device information message from the detectedsecondary device. The primary user device may determine capabilities ofthe secondary device based on the received device information message.In response to receiving a user request to access a content asset, theprimary user device may generate a content information message thatincludes a characteristic of the accessed primary content asset. Thecontent information message may then be transmitted to the secondarydevice.

In some embodiments, the primary user device may generate a messagetemplate based on the determined capabilities of the secondary device.The message template may reduce the amount of irrelevant information tobe transmitted to a secondary device. The primary user device maygenerate a content information message based on the message template. Insome embodiments, in response to transmitting a content informationmessage to a secondary device, the primary user device may receive afeedback information message from the secondary device that includesmetadata or timing information about a supplemental content asset thatis accessible on the secondary device.

It may be desirable to supplement one type of content asset with adifferent type of content asset. In some embodiments, the primary userdevice may select a secondary device to provide a supplemental contentasset based on the determined capabilities of the secondary device, andthe type of supplemental content asset desired.

In some embodiments, the primary user device may generate asynchronization information message that includes information comprisingmetadata or timing information used to coordinate playback of theprimary content asset and supplemental content asset. The primary userdevice may transmit the generated synchronization information message tothe secondary device.

In some embodiments, a characteristic of a content asset may includemetadata, timing information, user profile information, mood informationor feature information. The feature information may include featurevectors, audio features or visual features.

In some embodiments, a secondary device may receive an electroniccommunication from the primary user device, including information abouta primary content asset accessed at the primary user device. Thesecondary device may then access a database to search for contentlistings that are relevant to the primary content asset. The relevancymay be determined by comparing a characteristic of a content listingaccessed from the database to a characteristic of the primary contentasset.

In some embodiments, the secondary device may generate the database ofcontent listings based on content that is accessible to the secondarydevice. In some embodiments, the secondary device is a nexus device thatmay communicate with one or more slave secondary devices. The nexusdevice may receive information about content accessible to the slavesecondary device and update the database of content listings to includeinformation about the content accessible to the slave secondary device.The content available to the slave secondary device may be provided as asupplemental content asset to the primary content asset accessed by theprimary user device.

In some embodiments, the secondary device may determine if a contentlisting accessed from the database is relevant to the primary contentasset by analyzing or comparing information such as metadata or featureinformation. If the feature information of the primary content asset andcontent listing are different types, the feature information may bepreprocessed and normalized into formats that may be compared.

In some embodiments, a secondary device may retrieve information from asocial network account of a user of the primary user device from a webserver, and determine relevancy of a content listing based in part onthe mood information. In some embodiments, the secondary device maydetermine relevancy of a content listing based on mood informationassociated with the primary content asset accessed on the primary userdevice.

In some embodiments, a primary user device may communicate with a webserver to post mood information to a social network account of a user ofthe primary user device. The mood information to be posted may bereceived as input to the primary user device from the user, ordetermined from a primary content asset accessed by the primary userdevice.

In some embodiments, the supplemental content asset may be anadvertisement related to the primary content asset. This may be a formof context-based advertising based on the characteristics of the primarycontent asset accessed by the user. In some embodiments, thesupplemental content asset may be a related content that enhances thelistening or viewing experience of the user accessing the content. Forexample, in response to play back of a song on a portable mp3 player,television equipment may display a slide show of related images. In someembodiments, a secondary device may receive a user request to purchase asupplemental content asset from a content source.

In some embodiments, several content may be used to supplement the playback of an accessed primary content asset. For example, several contentfiles may be added to a playlist that supplements a primary contentasset.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an illustrative interactive media guidance applicationdisplay screen in accordance with some embodiments of the presentdisclosure;

FIG. 2 shows another illustrative interactive media guidance applicationdisplay screen in accordance with some embodiments of the presentdisclosure;

FIG. 3 illustrates an example of a user device that may provide asupplemental content asset in accordance with some embodiments of thepresent disclosure;

FIG. 4 illustrates an example of a system that provides supplementalcontent assets in accordance with some embodiments of the presentdisclosure;

FIG. 5 illustrates examples of user devices that may provide asupplemental content asset in accordance with some embodiments of thepresent disclosure;

FIG. 6 illustrates a flowchart of steps for providing to a primarycontent asset accessed on a primary user device, a supplemental contentasset accessed on a secondary device in accordance with some embodimentsof the present disclosure;

FIG. 7 illustrates a flowchart of steps, performed by a primary userdevice, that describes a process for providing a supplemental contentasset from a secondary device to a primary content asset accessed on aprimary user device in accordance with some embodiments of the presentdisclosure;

FIG. 8 illustrates a flowchart of steps, performed by a secondarydevice, that describes a process for providing a supplemental contentasset from a secondary device to a primary content asset accessed on aprimary user device in accordance with some embodiments of the presentdisclosure;

FIG. 9 illustrates an example of a display screen that may be shown on aprimary user device that accesses a primary content asset that may besupplemented, or shown on a secondary device that provides asupplemental content asset in accordance with some embodiments of thepresent disclosure;

FIG. 10 illustrates an example of a display screen that may be shown toa user on a primary user device or secondary device to configure theproviding of a supplemental content asset in accordance with someembodiments of the present disclosure;

FIG. 11 illustrates an example of a display screen that may be shown toa user on a primary user device or secondary device to set up theproviding of a supplemental content asset in accordance with someembodiments of the present disclosure;

FIG. 12 illustrates an example of a display screen that may be shown toa user on a primary user device or secondary device to select asupplemental content asset in accordance with some embodiments of thepresent disclosure;

FIG. 13 illustrates an example of a display screen that may be shown toa user on a secondary device displaying a supplemental content asset inaccordance with some embodiments of the present disclosure;

FIG. 14 illustrates an example of a display screen that may be shown toa user on a secondary device that displays a supplemental content assetin accordance with some embodiments of the present disclosure.

FIG. 15 illustrates an example of a data structure of an informationmessage for transmitting information between a primary user device andsecondary device in accordance with some embodiments of the disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS

The amount of content available to users in any given content deliverysystem can be substantial. Consequently, many users desire a form ofmedia guidance through an interface that allows users to efficientlynavigate content selections and easily identify content that they maydesire. An application that provides such guidance is referred to hereinas an interactive media guidance application or, sometimes, a mediaguidance application or a guidance application.

Interactive media guidance applications may take various forms dependingon the content for which they provide guidance. One typical type ofmedia guidance application is an interactive television program guide.Interactive television program guides (sometimes referred to aselectronic program guides) are well-known guidance applications that,among other things, allow users to navigate among and locate many typesof content. As referred to herein, the term “content” should beunderstood to mean an electronically consumable user asset, such astelevision programming, as well as pay-per-view programs, on-demandprograms (as in video-on-demand (VOD) systems), Internet content (e.g.,streaming content, downloadable content, Webcasts, etc.), video clips,audio, content information, pictures, rotating images, documents,playlists, websites, articles, books, electronic books, blogs,advertisements, chat sessions, social media, applications, games, and/orany other media or multimedia and/or combination of the same. Guidanceapplications also allow users to navigate among and locate content. Asreferred to herein, the term “multimedia” should be understood to meancontent that utilizes at least two different content forms describedabove, for example, text, audio, images, video, or interactivity contentforms. Content may be recorded, played, displayed or accessed by userequipment devices, but can also be part of a live performance.

With the advent of the Internet, mobile computing, and high-speedwireless networks, users are accessing media on user equipment deviceson which they traditionally did not. As referred to herein, the phrase“user equipment device,” “user equipment,” “user device,” “electronicdevice,” “electronic equipment,” “media equipment device,” or “mediadevice” should be understood to mean any device for accessing thecontent described above, such as a television, a Smart TV, a set-topbox, an integrated receiver decoder (IRD) for handling satellitetelevision, a digital storage device, a digital media receiver (DMR), adigital media adapter (DMA), a streaming media device, a DVD player, aDVD recorder, a connected DVD, a local media server, a BLU-RAY player, aBLU-RAY recorder, a personal computer (PC), a laptop computer, a tabletcomputer, a WebTV box, a personal computer television (PC/TV), a PCmedia server, a PC media center, a hand-held computer, a stationarytelephone, a personal digital assistant (PDA), a mobile telephone, aportable video player, a portable music player, a portable gamingmachine, a smart phone, or any other television equipment, computingequipment, or wireless device, and/or combination of the same. In someembodiments, the user equipment device may have a front facing screenand a rear facing screen, multiple front screens, or multiple angledscreens. In some embodiments, the user equipment device may have a frontfacing camera and/or a rear facing camera. On these user equipmentdevices, users may be able to navigate among and locate the same contentavailable through a television. Consequently, media guidance may beavailable on these devices, as well. The guidance provided may be forcontent available only through a television, for content available onlythrough one or more of other types of user equipment devices, or forcontent available both through a television and one or more of the othertypes of user equipment devices. The media guidance applications may beprovided as on-line applications (i.e., provided on a web-site), or asstand-alone applications or clients on user equipment devices. Variousdevices and platforms that may implement media guidance applications aredescribed in more detail below.

One of the functions of the media guidance application is to providemedia guidance data to users. As referred to herein, the phrase, “mediaguidance data” or “guidance data” should be understood to mean any datarelated to content, such as media listings, media-related information(e.g., broadcast times, broadcast channels, titles, descriptions,ratings information (e.g., parental control ratings, critic's ratings,etc.), genre or category information, actor information, logo data forbroadcasters' or providers' logos, etc.), media format (e.g., standarddefinition, high definition, 3D, etc.), advertisement information (e.g.,text, images, media clips, etc.), on-demand information, blogs,websites, and any other type of guidance data that is helpful for a userto navigate among and locate desired content selections.

FIGS. 1-2 show illustrative display screens that may be used to providemedia guidance data. The display screens shown in FIGS. 1-2 and 9-14 maybe implemented on any suitable user equipment device or platform. Whilethe displays of FIGS. 1-2 and 9-14 are illustrated as full screendisplays, they may also be fully or partially overlaid over contentbeing displayed. A user may indicate a desire to access contentinformation by selecting a selectable option provided in a displayscreen (e.g., a menu option, a listings option, an icon, a hyperlink,etc.) or pressing a dedicated button (e.g., a GUIDE button) on a remotecontrol or other user input interface or device. In response to theuser's indication, the media guidance application may provide a displayscreen with media guidance data organized in one of several ways, suchas by time and channel in a grid, by time, by channel, by source, bycontent type, by category (e.g., movies, sports, news, children, orother categories of programming), or other predefined, user-defined, orother organization criteria. The organization of the media guidance datais determined by guidance application data. As referred to herein, thephrase, “guidance application data” should be understood to mean dataused in operating the guidance application, such as program information,guidance application settings, user preferences, or user profileinformation.

FIG. 1 shows illustrative grid program listings display 100 arranged bytime and channel that also enables access to different types of contentin a single display. Display 100 may include grid 102 with: (1) a columnof channel/content type identifiers 104, where each channel/content typeidentifier (which is a cell in the column) identifies a differentchannel or content type available; and (2) a row of time identifiers106, where each time identifier (which is a cell in the row) identifiesa time block of programming. Grid 102 also includes cells of programlistings, such as program listing 108, where each listing provides thetitle of the program provided on the listing's associated channel andtime. With a user input device, a user can select program listings bymoving highlight region 110. Information relating to the program listingselected by highlight region 110 may be provided in program informationregion 112. Region 112 may include, for example, the program title, theprogram description, the time the program is provided (if applicable),the channel the program is on (if applicable), the program's rating, andother desired information.

In addition to providing access to linear programming (e.g., contentthat is scheduled to be transmitted to a plurality of user equipmentdevices at a predetermined time and is provided according to aschedule), the media guidance application also provides access tonon-linear programming (e.g., content accessible to a user equipmentdevice at any time and is not provided according to a schedule).Non-linear programming may include content from different contentsources including on-demand content (e.g., VOD), Internet content (e.g.,streaming media, downloadable media, etc.), locally stored content(e.g., content stored on any user equipment device described above orother storage device), or other time-independent content. On-demandcontent may include movies or any other content provided by a particularcontent provider (e.g., HBO On Demand providing “The Sopranos” and “CurbYour Enthusiasm”). HBO ON DEMAND is a service mark owned by Time WarnerCompany L. P. et al. and THE SOPRANOS and CURB YOUR ENTHUSIASM aretrademarks owned by the Home Box Office, Inc. Internet content mayinclude web events, such as a chat session or Webcast, or contentavailable on-demand as streaming content or downloadable content throughan Internet web site or other Internet access (e.g. FTP).

Grid 102 may provide media guidance data for non-linear programmingincluding on-demand listing 114, recorded content listing 116, andInternet content listing 118. A display combining media guidance datafor content from different types of content sources is sometimesreferred to as a “mixed-media” display. Various permutations of thetypes of media guidance data that may be displayed that are differentfrom display 100 may be based on user selection or guidance applicationdefinition (e.g., a display of only recorded and broadcast listings,only on-demand and broadcast listings, etc.). As illustrated, listings114, 116, and 118 are shown as spanning the entire time block displayedin grid 102 to indicate that selection of these listings may provideaccess to a display dedicated to on-demand listings, recorded listings,or Internet listings, respectively. In some embodiments, listings forthese content types may be included directly in grid 102. Additionalmedia guidance data may be displayed in response to the user selectingone of the navigational icons 120. (Pressing an arrow key on a userinput device may affect the display in a similar manner as selectingnavigational icons 120.)

Display 100 may also include video region 122, advertisement 124, andoptions region 126. Video region 122 may allow the user to view and/orpreview programs that are currently available, will be available, orwere available to the user. The content of video region 122 maycorrespond to, or be independent from, one of the listings displayed ingrid 102. Grid displays including a video region are sometimes referredto as picture-in-guide (PIG) displays. PIG displays and theirfunctionalities are described in greater detail in Satterfield et al.U.S. Pat. No. 6,564,378, issued May 13, 2003 and Yuen et al. U.S. Pat.No. 6,239,794, issued May 29, 2001, which are hereby incorporated byreference herein in their entireties. PIG displays may be included inother media guidance application display screens of the embodimentsdescribed herein.

Advertisement 124 may provide an advertisement for content that,depending on a viewer's access rights (e.g., for subscriptionprogramming), is currently available for viewing, will be available forviewing in the future, or may never become available for viewing, andmay correspond to or be unrelated to one or more of the content listingsin grid 102. Advertisement 124 may also be for products or servicesrelated or unrelated to the content displayed in grid 102. Advertisement124 may be selectable and provide further information about content,provide information about a product or a service, enable purchasing ofcontent, a product, or a service, provide content relating to theadvertisement, etc. Advertisement 124 may be targeted based on a user'sprofile/preferences, monitored user activity, the type of displayprovided, or on other suitable targeted advertisement bases.

While advertisement 124 is shown as rectangular or banner shaped,advertisements may be provided in any suitable size, shape, and locationin a guidance application display. For example, advertisement 124 may beprovided as a rectangular shape that is horizontally adjacent to grid102. This is sometimes referred to as a panel advertisement. Inaddition, advertisements may be overlaid over content or a guidanceapplication display or embedded within a display. Advertisements mayalso include text, images, rotating images, video clips, or other typesof content described above. Advertisements may be stored in a userequipment device having a guidance application, in a database connectedto the user equipment, in a remote location (including streaming mediaservers), or on other storage means, or a combination of theselocations. Providing advertisements in a media guidance application isdiscussed in greater detail in, for example, Knudson et al., U.S. PatentApplication Publication No. 2003/0110499, filed Jan. 17, 2003; Ward, IIIet al. U.S. Pat. No. 6,756,997, issued Jun. 29, 2004; and Schein et al.U.S. Pat. No. 6,388,714, issued May 14, 2002, which are herebyincorporated by reference herein in their entireties. It will beappreciated that advertisements may be included in other media guidanceapplication display screens of the embodiments described herein.

Options region 126 may allow the user to access different types ofcontent, media guidance application displays, and/or media guidanceapplication features. Options region 126 may be part of display 100 (andother display screens described herein), or may be invoked by a user byselecting an on-screen option or pressing a dedicated or assignablebutton on a user input device. The selectable options within optionsregion 126 may concern features related to program listings in grid 102or may include options available from a main menu display. Featuresrelated to program listings may include searching for other air times orways of receiving a program, recording a program, enabling seriesrecording of a program, setting program and/or channel as a favorite,purchasing a program, or other features. Options available from a mainmenu display may include search options, VOD options, parental controloptions, Internet options, cloud-based options, device synchronizationoptions, second screen device options, options to access various typesof media guidance data displays, options to subscribe to a premiumservice, options to edit a user's profile, options to access a browseoverlay, or other options.

The media guidance application may be personalized based on a user'spreferences. A personalized media guidance application allows a user tocustomize displays and features to create a personalized “experience”with the media guidance application. This personalized experience may becreated by allowing a user to input these customizations and/or by themedia guidance application monitoring user activity to determine varioususer preferences. Users may access their personalized guidanceapplication by logging in or otherwise identifying themselves to theguidance application. Customization of the media guidance applicationmay be made in accordance with a user profile. The customizations mayinclude varying presentation schemes (e.g., color scheme of displays,font/size of text, etc.), aspects of content listings displayed (e.g.,only HDTV or only 3D programming, user-specified broadcast channelsbased on favorite channel selections, re-ordering the display ofchannels, recommended content, etc.), desired recording features (e.g.,recording or series recordings for particular users, recording quality,etc.), parental control settings, customized presentation of Internetcontent (e.g., presentation of social content, e-mail, electronicallydelivered articles, etc.) and other desired customizations.

The media guidance application may allow a user to provide user profileinformation or may automatically compile user profile information. Themedia guidance application may, for example, monitor the content theuser accesses and/or other interactions the user may have with theguidance application. Additionally, the media guidance application mayobtain all or part of other user profiles that are related to aparticular user (e.g., from other web sites on the Internet the useraccesses, such as www.allrovi.com, from other media guidanceapplications the user accesses, from other interactive applications theuser accesses, from another user equipment device of the user, etc.),and/or obtain information about the user from other sources that themedia guidance application may access. As a result, a user can beprovided with a unified guidance application experience across theuser's different user equipment devices. This type of user experience isdescribed in greater detail below in connection with FIG. 4. Additionalpersonalized media guidance application features are described ingreater detail in Ellis et al., U.S. Patent Application Publication No.2005/0251827, filed Jul. 11, 2005, Boyer et al., U.S. Pat. No.7,165,098, issued Jan. 16, 2007, and Ellis et al., U.S. PatentApplication Publication No. 2002/0174430, filed Feb. 21, 2002, which arehereby incorporated by reference herein in their entireties.

User profile information may be defined as information that indicatescharacteristics of a particular user. User profile information mayinclude name, gender, age, race, preference, mood, status or any othersuitable information indicating characteristics of a user. Methods andsystems for using user profile information have been described in detailin U.S. Pat. No. 6,177,931, issued Jan. 23, 2011, which is herebyincorporated by reference herein in its entirety. Preference informationmay indicate favorite media related information. For example, preferenceinformation may include a user's favorite genre or category. Preferenceinformation may indicate a user's most commonly accessed primary contentasset or content assets. For example, preference information mayindicate that a user accesses a particular song or set of songs moreoften than any other songs on an mp3 player. User devices may use theuser profile information to search for a relevant supplemental contentasset.

Another display arrangement for providing media guidance is shown inFIG. 2. Video mosaic display 200 includes selectable options 202 forcontent information organized based on content type, genre, and/or otherorganization criteria. In display 200, television listings option 204 isselected, thus providing listings 206, 208, 210, and 212 as broadcastprogram listings. In display 200 the listings may provide graphicalimages including cover art, still images from the content, video clippreviews, live video from the content, or other types of content thatindicate to a user the content being described by the media guidancedata in the listing. Each of the graphical listings may also beaccompanied by text to provide further information about the contentassociated with the listing. For example, listing 208 may include morethan one portion, including media portion 214 and text portion 216.Media portion 214 and/or text portion 216 may be selectable to viewcontent in full-screen or to view information related to the contentdisplayed in media portion 214 (e.g., to view listings for the channelthat the video is displayed on).

The listings in display 200 are of different sizes (i.e., listing 206 islarger than listings 208, 210, and 212), but if desired, all thelistings may be the same size. Listings may be of different sizes orgraphically accentuated to indicate degrees of interest to the user orto emphasize certain content, as desired by the content provider orbased on user preferences. Various systems and methods for graphicallyaccentuating content listings are discussed in, for example, Yates, U.S.Patent Application Publication No. 2010/0153885, filed Dec. 29, 2005,which is hereby incorporated by reference herein in its entirety.

Users may access content and the media guidance application (and itsdisplay screens described above and below) from one or more of theiruser equipment devices. FIG. 3 shows a generalized embodiment ofillustrative user equipment device 300. The embodiment generalized byFIG. 3 may operate as a primary user device or a secondary device. Morespecific implementations of user equipment devices are discussed belowin connection with FIG. 4. User equipment device 300 may receive contentand data via input/output (hereinafter “I/O”) path 302. I/O path 302 mayprovide content (e.g., broadcast programming, on-demand programming,Internet content, content available over a local area network (LAN) orwide area network (WAN), and/or other content) and data to controlcircuitry 304, which includes processing circuitry 306, detectingcircuitry 307, and storage 308. Control circuitry 304 may be used tosend and receive commands, requests, and other suitable data using I/Opath 302. I/O path 302 may connect control circuitry 304 (andspecifically processing circuitry 306) to one or more communicationspaths (described below). I/O functions may be provided by one or more ofthese communications paths, but are shown as a single path in FIG. 3 toavoid overcomplicating the drawing.

Control circuitry 304 may be based on any suitable processing circuitrysuch as processing circuitry 306. As referred to herein, processingcircuitry should be understood to mean circuitry based on one or moremicroprocessors, microcontrollers, digital signal processors,programmable logic devices, field-programmable gate arrays (FPGAs),application-specific integrated circuits (ASICs), etc., and may includea multi-core processor (e.g., dual-core, quad-core, hexa-core, or anysuitable number of cores) or supercomputer. In some embodiments,processing circuitry may be distributed across multiple separateprocessors or processing units, for example, multiples of the same typeof processing units (e.g., two Intel Core i7 processors) or multipledifferent processors (e.g., an Intel Core i5 processor and an Intel Corei7 processor). In some embodiments, control circuitry 304 executesinstructions for a media guidance application stored in memory (i.e.,storage 308).

In client-server based embodiments, control circuitry 304 may includecommunications circuitry suitable for communicating with a guidanceapplication server or other networks or servers. The instructions forcarrying out the above-mentioned functionality may be stored on theguidance application server. Communications circuitry may include acable modem, an integrated services digital network (ISDN) modem, adigital subscriber line (DSL) modem, a telephone modem, Ethernet card, awireless modem, a local area networking (LAN) Ethernet adapter, awireless LAN adapter, a wireless bluetooth adapter, USB port, orwireless USB port for communications with other equipment, or any othersuitable communications circuitry. Such communications may involve theInternet or any other suitable communications networks or paths (whichis described in more detail in connection with FIG. 4). In addition,communications circuitry may include circuitry that enables peer-to-peercommunication of user equipment devices, or communication of userequipment devices in locations remote from each other (described in moredetail below).

In some embodiments, control circuitry 304 may include detectingcircuitry 307 which may be capable of detecting and/or identifying auser or users without requiring the user or users to make anyaffirmative actions by using any suitable biometric recognitiontechnique, such as facial recognition, heat signature recognition, odorrecognition, scent recognition, body shape recognition, voicerecognition, behavioral recognition, or any other suitable biometricrecognition technique. In some embodiments detecting circuitry maydetect or identify a user device using wireless detection techniques, aswill be described in further detail below. For example, detectingcircuitry 307 may detect and identify users using these techniques whilethe users are within a measurable range to an electronic device. Themeasurable range may be limited by capability of the detecting circuitryto resolve or measure features. For example, the measurable range fromwhich a camera may resolve two spaced points may be limited based onoptical resolution of the camera. For example, the measurable range fromwhich a wireless antenna may be able to detect a received signal abovebackground noise may depend on the initial intensity of the transmittedsignal and an absorption coefficient of the surrounding transmissionmedium. In some embodiments, users may be detected and/or identifiedusing any other suitable biometric recognition technique that may insome embodiments require the users to be within a measurable range to anelectronic device, for example, iris recognition, retinal recognition,palm recognition, finger print recognition, or any other suitabletechnique.

Detecting circuitry 307, by using wireless detection techniques, mayalso be capable of detecting and/or identifying a user device (e.g., amobile device, such as an RFID device or mobile phone). Detectingcircuitry 307 may recognize and identify such a device using anysuitable means, for example, radio-frequency identification, Bluetooth,Wi-Fi, WiMax, Internet protocol, infrared signals, any other suitableInstitute for Electrical and Electronics Engineers (IEEE), industrial,or proprietary communication standards, or any other suitableelectronic, optical, or auditory communication means. In someembodiments, detecting circuitry may detect a user. For example,detecting circuitry 307 may determine that a user is within a detectionregion of an electronic device, identify the user, and add the user to alist of active users at the electronic device.

The detection and identification of users as described herein does notrequire any affirmative action on the part of the user beyond, in someembodiments, the configuration of such methods and systems. For example,any detection and identification of users is done automatically byelectronic devices. In some embodiments, detecting circuitry may detecta user device. For example, detecting circuitry 307 may determine byusing wireless detection techniques that a user device is within awireless detection region, and add the user device to a list of userdevices at the electronic device. In some embodiments, detectingcircuitry 307 may detect a user using a biometric technique, and inresponse to detecting a user, detect and identify a user device usingwireless techniques. For example, a secondary device may detect a user,and query information about primary content assets on the primary userdevice. For example, detecting circuitry 307 of an electronic device maydetect a user on a biometric device such as a camera, and in response todetecting the user, wirelessly detect a primary user devicecorresponding to the detected user. If a primary user device isdetected, the secondary device may communicate with primary device toprovide supplemental content. Further embodiments and implementationswill be discussed in greater detail below.

Detecting circuitry 307 may include any suitable hardware and/orsoftware to perform detection and identification operations. Forexample, detecting circuitry 307 may include infrared, optical, and/orradio-frequency receivers and/or transmitters. Detecting circuitry 307may additionally, or alternatively, include one or more microphonesand/or cameras to detect audible and/or visual information,respectively. The microphone may be capable of receiving sounds withinthe audible range and/or outside the audible range. The camera may becapable of capturing information within the visual spectrum and/oroutside the visual spectrum. For example, the camera may be able tocapture infrared information, ultraviolet information, or any othersuitable type of information. Detecting circuitry 307 may additionally,or alternatively, include palm, fingerprint, and/or retinal readers fordetecting and/or identifying users. In some embodiments, detectingcircuitry may communicate to processing circuitry 306 and/or storage 308various detection and/or identification mechanisms indicating whether auser is detected and/or identified at a particular device.

In some embodiments, detecting circuitry 307 may use any suitable methodto determine the distance, trajectory, and/or location a user is at inrelation to an electronic device. For example, an electronic device mayuse received signal strength indication (RSSI) from a user's mobiledevice to determine the distance the user is to the electronic device.For example, RSSI values may be triangulated to determine a user'slocation. The electronic device may also use, for example, triangulationand/or time difference of arrival determination of appropriateinformation to determine a user's location in relation to an electronicdevice. For example, time difference of arrival values of soundsemanating from a user may be determined. In some embodiments, anysuitable image processing, video processing, and/or computer visiontechnique may be used to determine a user's distance, trajectory, and/orlocation in relation to an electronic device. A user's distance,trajectory, and/or location in relation to an electronic device may bedetermined using any suitable method.

In some embodiments, processing circuitry 306 may determine a locationbased on global positioning system (GPS) measurements, or, in the caseof cellular telephones, measurements based on cell-tower signals, doneby detecting circuitry 307. Processing circuitry 306 may use thesemeasurements to determine location coordinates which may be transmittedto other electronic devices.

Memory may be an electronic storage device provided as storage 308 thatis part of control circuitry 304. As referred to herein, the phrase“electronic storage device” or “storage device” should be understood tomean any device for storing electronic data, computer software,database, or firmware, such as random-access memory, read-only memory,nonvolatile memory, hard drives, optical drives, digital video disc(DVD) recorders, compact disc (CD) recorders, BLU-RAY disc (BD)recorders, BLU-RAY 3D disc recorders, digital video recorders (DVRs,sometimes called a personal video recorders, or PVRs), solid statedevices, quantum storage devices, gaming consoles, gaming media, or anyother suitable fixed or removable storage devices, and/or anycombination of the same. Storage 308 may be used to store various typesof content described herein as well as media guidance information,described above, and guidance application data, described above. Storage308 may be used to store a library of content files, a database ofcontent files, a database of content listings, or any other suitableinformation. A content listing may include information about a contentfile. Nonvolatile memory may also be used (e.g., to launch a boot-uproutine and other instructions). Cloud-based storage, described inrelation to FIG. 4, may be used to supplement storage 308 or instead ofstorage 308.

Control circuitry 304 may include video generating circuitry and tuningcircuitry, such as one or more analog tuners, one or more MPEG-2decoders or other digital decoding circuitry, high-definition tuners, orany other suitable tuning or video circuits or combinations of suchcircuits. Encoding circuitry (e.g., for converting over-the-air, analog,or digital signals to MPEG signals for storage) may also be provided.Control circuitry 304 may also include scaler circuitry for upconvertingand downconverting content into the preferred output format of the userequipment 300. Circuitry 304 may also include digital-to-analogconverter circuitry and analog-to-digital converter circuitry forconverting between digital and analog signals. The tuning and encodingcircuitry may be used by the user equipment device to receive and todisplay, to play, or to record content. The tuning and encodingcircuitry may also be used to receive guidance data. The circuitrydescribed herein, including for example, the tuning, video generating,encoding, decoding, encrypting, decrypting, scaler, and analog/digitalcircuitry, may be implemented using software running on one or moregeneral purpose or specialized processors. Multiple tuners may beprovided to handle simultaneous tuning functions (e.g., watch and recordfunctions, picture-in-picture (PIP) functions, multiple-tuner recording,etc.). If storage 308 is provided as a separate device from userequipment 300, the tuning and encoding circuitry (including multipletuners) may be associated with storage 308.

A user may send instructions to control circuitry 304 using user inputinterface 310. User input interface 310 may be any suitable userinterface, such as a remote control, mouse, trackball, keypad, keyboard,touch screen, touchpad, stylus input, joystick, voice recognitioninterface, or other user input interfaces. User input interface 310 maycommunicate with one or more media devices. In some embodiments, userinput interface 310 may be a remote control with pressure-sensitivebuttons. Control circuitry 304 may recognize different amounts ofpressure applied to the pressure-sensitive buttons and adjust navigationthrough, for example, the guidance application. For example, the amountof pressure applied may adjust the speed of scrolling through guidanceapplication listings, such that when less pressure is applied, thelistings scroll more slowly and when more pressure is applied, thelistings scroll more quickly. The pressure-sensitive buttons may controlany portion of the guidance application or an electronic device in anysuitable manner.

In some embodiments, user input interface 310 may communicate with mediadevices using a communications path as described in reference to FIG. 4.For example, user input interface 310 that may be incorporated into auser's mobile phone may communicate with media devices using a wirelesscommunications path. In some embodiments, user input interface 310 maycommunicate with an intermediate device that may convert any suitablecommunication means to any other suitable communication means. Forexample, a user may attempt to scroll through a guidance applicationusing the user's mobile phone incorporated with user input interface310. User input interface 310 may output wireless packets to communicatewith user's scroll instructions. An intermediate device may receive thewireless packets and, in turn, transmit infrared packets to controlcircuitry 304.

In some embodiments, user input interface 310 may store, transmit,and/or receive information associated with and/or identifying aparticular user or users. This information may be used by detectingcircuitry 307 to detect and/or identify that the user associated withthe information is within a detection region of an electronic device.The user may then be added to a list of active users at the electronicdevice and/or logged into the electronic device.

Display 312 may be provided as a stand-alone device or integrated withother elements of user equipment device 300. Display 312 may be one ormore of a monitor, a television, a liquid crystal display (LCD) for amobile device, or any other suitable equipment for displaying visualimages. In some embodiments, display 312 may be HDTV-capable. In someembodiments, display 312 may be a 3D display, and the interactive mediaguidance application and any suitable content may be displayed in 3D. Avideo card or graphics card may generate the output to the display 312.The video card may offer various functions such as accelerated renderingof 3D scenes and 2D graphics, MPEG-2/MPEG-4 decoding, TV output, or theability to connect multiple monitors. The video card may be anyprocessing circuitry described above in relation to control circuitry304. The video card may be integrated with the control circuitry 304.Speakers 314 may be provided as integrated with other elements of userequipment device 300 or may be stand-alone units. The audio component ofvideos and other content displayed on display 312 may be played throughspeakers 314. In some embodiments, the audio may be distributed to areceiver (not shown), which processes and outputs the audio via speakers314.

In some embodiments, control circuitry 304 may retrieve one or more userselected programs that are provided by a content source. Controlcircuitry 304 may instruct a storage device to store the contentprovided by the content source during the scheduled broadcast time(e.g., having a scheduled start time and a scheduled end time) of theprogram selected for recording.

The guidance application may be implemented using any suitablearchitecture. For example, it may be a stand-alone application whollyimplemented on user equipment device 300. In such an approach,instructions of the application are stored locally, and data for use bythe application is downloaded on a periodic basis (e.g., from anout-of-band feed, from an Internet resource, or using another suitableapproach). In some embodiments, the media guidance application is aclient-server based application. Data for use by a thick or thin clientimplemented on user equipment device 300 is retrieved on-demand byissuing requests to a server remote to the user equipment device 300. Inone example of a client-server based guidance application, controlcircuitry 304 runs a web browser that interprets web pages provided by aremote server.

In some embodiments, the media guidance application is downloaded andinterpreted or otherwise run by an interpreter or virtual machine (runby control circuitry 304). In some embodiments, the guidance applicationmay be encoded in the ETV Binary Interchange Format (EBIF), received bycontrol circuitry 304 as part of a suitable feed, and interpreted by auser agent running on control circuitry 304. For example, the guidanceapplication may be an EBIF application. In some embodiments, theguidance application may be defined by a series of JAVA-based files thatare received and run by a local virtual machine or other suitablemiddleware executed by control circuitry 304. In some of suchembodiments (e.g., those employing MPEG-2 or other digital mediaencoding schemes), the guidance application may be, for example, encodedand transmitted in an MPEG-2 object carousel with the MPEG audio andvideo packets of a program.

User equipment device 300 of FIG. 3 can be implemented in system 400 ofFIG. 4 as user television equipment 402, user computer equipment 404,wireless user communications device 406, or any other type of userequipment suitable for accessing content, such as a non-portable gamingmachine. For simplicity, these devices may be referred to hereincollectively as user equipment or user equipment devices, and may besubstantially similar to user equipment devices described above. Userequipment devices, on which a media guidance application may beimplemented, may function as a standalone device or may be part of anetwork of devices. Various network configurations of devices may beimplemented and are discussed in more detail below.

A user equipment device utilizing at least some of the system featuresdescribed above in connection with FIG. 3 may not be classified solelyas user television equipment 402, user computer equipment 404, or awireless user communications device 406. For example, user televisionequipment 402 may, like some user computer equipment 404, beInternet-enabled, allowing for access to Internet content, while usercomputer equipment 404 may, like some television equipment 402, includea tuner allowing for access to television programming. The mediaguidance application may have the same layout on various different typesof user equipment or may be tailored to the display capabilities of theuser equipment. For example, on user computer equipment 404, theguidance application may be provided as a web site accessed by a webbrowser. In another example, the guidance application may be scaled downfor wireless user communications devices 406.

In system 400, there is typically more than one of each type of userequipment device but only one of each is shown in FIG. 4 to avoidovercomplicating the drawing. In addition, each user may utilize morethan one type of user equipment device and also more than one of eachtype of user equipment device.

In some embodiments, a user equipment device (e.g., user televisionequipment 402, user computer equipment 404, wireless user communicationsdevice 406) may be referred to as a “second screen device.” For example,a second screen device may supplement content presented on a first userequipment device. The content presented on the second screen device maybe any suitable content that supplements the content presented on thefirst device. In some embodiments, the second screen device provides aninterface for adjusting settings and display preferences of the firstdevice. In some embodiments, the second screen device is configured forinteracting with other second screen devices or for interacting with asocial network. The second screen device can be located in the same roomas the first device, a different room from the first device but in thesame house or building, or in a different building from the firstdevice.

The user may also set various settings to maintain consistent mediaguidance application settings across in-home devices and remote devices.Settings include those described herein, as well as channel and programfavorites, programming preferences that the guidance applicationutilizes to make programming recommendations, display preferences, andother desirable guidance settings. For example, if a user sets a channelas a favorite on, for example, the web site www.allrovi.com on theirpersonal computer at their office, the same channel would appear as afavorite on the user's in-home devices (e.g., user television equipmentand user computer equipment) as well as the user's mobile devices, ifdesired. Therefore, changes made on one user equipment device can changethe guidance experience on another user equipment device, regardless ofwhether they are the same or a different type of user equipment device.In addition, the changes made may be based on settings input by a user,as well as user activity monitored by the guidance application.

The user equipment devices may be coupled to communications network 414.Namely, user television equipment 402, user computer equipment 404, andwireless user communications device 406 are coupled to communicationsnetwork 414 via communications paths 408, 410, and 412, respectively.Communications network 414 may be one or more networks including theInternet, a mobile phone network, mobile voice or data network (e.g., a4G or LTE network), cable network, public switched telephone network, orother types of communications network or combinations of communicationsnetworks. Paths 408, 410, and 412 may separately or together include oneor more communications paths, such as a satellite path, a fiber-opticpath, a cable path, a path that supports Internet communications (e.g.,IPTV), free-space connections (e.g., for broadcast or other wirelesssignals), or any other suitable wired or wireless communications path orcombination of such paths. Path 412 is drawn with dotted lines toindicate that in the exemplary embodiment shown in FIG. 4 it is awireless path and paths 408 and 410 are drawn as solid lines to indicatethey are wired paths (although these paths may be wireless paths, ifdesired). Communications with the user equipment devices may be providedby one or more of these communications paths, but are shown as a singlepath in FIG. 4 to avoid overcomplicating the drawing.

Although communications paths are not drawn between user equipmentdevices, these devices may communicate directly with each other viacommunication paths, such as those described above in connection withpaths 408, 410, and 412, as well other short-range point-to-pointcommunication paths, such as USB cables, IEEE 1394 cables, ethernetcables, wireless paths (e.g., Bluetooth, infrared, IEEE 802-11x,wireless USB, etc.), or other short-range communication via wired orwireless paths. BLUETOOTH is a certification mark owned by BluetoothSIG, INC. The user equipment devices may also communicate with eachother directly through an indirect path via communications network 414.

System 400 includes content source 416 and media guidance data source418 coupled to communications network 414 via communication paths 420and 422, respectively. Paths 420 and 422 may include any of thecommunication paths described above in connection with paths 408, 410,and 412. Communications with the content source 416 and media guidancedata source 418 may be exchanged over one or more communications paths,but are shown as a single path in FIG. 4 to avoid overcomplicating thedrawing. In addition, there may be more than one of each of contentsource 416 and media guidance data source 418, but only one of each isshown in FIG. 4 to avoid overcomplicating the drawing. (The differenttypes of each of these sources are discussed below.) If desired, contentsource 416 and media guidance data source 418 may be integrated as onesource device. Although communications between sources 416 and 418 withuser equipment devices 402, 404, and 406 are shown as throughcommunications network 414, in some embodiments, sources 416 and 418 maycommunicate directly with user equipment devices 402, 404, and 406 viacommunication paths (not shown) such as those described above inconnection with paths 408, 410, and 412.

System 400 may include a web server 430 that may host a website, socialnetwork, an Internet database of information, or any other suitableonline service. User equipment 402 may receive through communicationsnetwork 414 information (e.g., in real-time or during predefined timeintervals) from one or more online services hosted by web server 430.Examples of social networks include Facebook, MySpace, and Google+.Facebook is a trademark owned by Facebook Incorporated. MySpace is atrademark owned by Specific Media LLC. Google+ is a trademark owned byGoogle Incorporated. For example, mood information may be posted to andretrieved from a social network account of a user hosted on a webserver. The mood information may be used by processing circuitry of aprimary user device or by processing circuitry of a secondary device tomake recommendations of content relevant to the mood information. Insome implementations, processing circuitry 306 may query an onlineservice by issuing a communication to web server 430. In particular,processing circuitry 306 may query the online service for moodinformation associated with a user.

Content source 416 may include one or more types of content distributionequipment including a television distribution facility, cable systemheadend, satellite distribution facility, programming sources (e.g.,television broadcasters, such as NBC, ABC, HBO, etc.), intermediatedistribution facilities and/or servers, Internet providers, on-demandmedia servers, and other content providers. NBC is a trademark owned bythe National Broadcasting Company, Inc., ABC is a trademark owned by theABC, INC., and HBO is a trademark owned by the Home Box Office, Inc.Content source 416 may be the originator of content (e.g., a televisionbroadcaster, a Webcast provider, etc.) or may not be the originator ofcontent (e.g., an on-demand content provider, an Internet provider ofcontent of broadcast programs for downloading, etc.). Content source 416may include cable sources, satellite providers, on-demand providers,Internet providers, over-the-top content providers, or other providersof content. Content source 416 may also include a remote media serverused to store different types of content (including video contentselected by a user), in a location remote from any of the user equipmentdevices. Systems and methods for remote storage of content, andproviding remotely stored content to user equipment are discussed ingreater detail in connection with Ellis et al., U.S. Pat. No. 7,761,892,issued Jul. 20, 2010, which is hereby incorporated by reference hereinin its entirety.

Content source 416 may also include a local media server used to storedifferent types of content, in a location near the user devices. In someembodiments, content source 416 and media guidance data source 418 maybe integrated into a user device. For example, a user televisionequipment 402, such as a set-top box, may contain a media server thatincludes content source 416 and media guidance data source 418.

Media guidance data source 418 may provide media guidance data, such asthe media guidance data described above. Media guidance application datamay be provided to the user equipment devices using any suitableapproach. In some embodiments, the guidance application may be astand-alone interactive television program guide that receives programguide data via a data feed (e.g., a continuous feed or trickle feed, ordata in the vertical blanking interval of a channel). Program scheduledata and other guidance data may be provided to the user equipment on atelevision channel sideband, in the vertical blanking interval of atelevision channel, using an in-band digital signal, using anout-of-band digital signal, or by any other suitable data transmissiontechnique. Program schedule data and other media guidance data may beprovided to user equipment on multiple analog or digital televisionchannels.

In some embodiments, guidance data from media guidance data source 418may be provided to users' equipment using a client-server approach. Forexample, a user equipment device may pull media guidance data from aserver, or a server may push media guidance data to a user equipmentdevice. In some embodiments, a guidance application client residing onthe user's equipment may initiate sessions with source 418 to obtainguidance data when needed, e.g., when the guidance data is out of dateor when the user equipment device receives a request from the user toreceive data. Media guidance may be provided to the user equipment withany suitable frequency (e.g., continuously, daily, a user-specifiedperiod of time, a system-specified period of time, in response to arequest from user equipment, etc.). Media guidance data source 418 mayprovide user equipment devices 402, 404, and 406 the media guidanceapplication itself or software updates for the media guidanceapplication.

Media guidance applications may be, for example, stand-aloneapplications implemented on user equipment devices. For example, themedia guidance application may be implemented as software or a set ofexecutable instructions which may be stored in storage 308, and executedby control circuitry 304 of a user equipment device 300. In someembodiments, media guidance applications may be client-serverapplications where only a client component of the application resides onthe user equipment device, and server application resides on a remoteserver. For example, media guidance applications may be implementedpartially as a client application on control circuitry 304 of userequipment device 300 and partially on a remote server as a serverapplication (e.g., media guidance data source 418) running on controlcircuitry of the remote server. When executed by control circuitry ofthe remote server (such as media guidance data source 418), the mediaguidance application may instruct the control circuitry to generate theguidance application displays and transmit the generated displays to theuser equipment devices. The server application may instruct the controlcircuitry of the media guidance data source 418 to transmit data forstorage on the user equipment. The client application may instructcontrol circuitry of the receiving user equipment to generate theguidance application displays.

Content and/or media guidance data delivered to user equipment devices402, 404, and 406 may be over-the-top (OTT) content. OTT contentdelivery allows Internet-enabled user devices, including any userequipment device described above, to receive content that is transferredover the Internet, including any content described above, in addition tocontent received over cable or satellite connections. OTT content isdelivered via an Internet connection provided by an Internet serviceprovider (ISP), but a third party distributes the content. The ISP maynot be responsible for the viewing abilities, copyrights, orredistribution of the content, and may only transfer IP packets providedby the OTT content provider. Examples of OTT content providers includeYOUTUBE, NETFLIX, and HULU, which provide audio and video via IPpackets. Youtube is a trademark owned by Google Inc., Netflix is atrademark owned by Netflix Inc., and Hulu is a trademark owned by Hulu,LLC. OTT content providers may additionally or alternatively providemedia guidance data described above. In addition to content and/or mediaguidance data, providers of OTT content can distribute media guidanceapplications (e.g., web-based applications or cloud-based applications),or the content can be displayed by media guidance applications stored onthe user equipment device.

Media guidance system 400 is intended to illustrate a number ofapproaches, or network configurations, by which user equipment devicesand sources of content and guidance data may communicate with each otherfor the purpose of accessing content and providing media guidance. Theembodiments described herein may be applied in any one or a subset ofthese approaches, or in a system employing other approaches fordelivering content and providing media guidance. The following fourapproaches provide specific illustrations of the generalized example ofFIG. 4.

In one approach, user equipment devices may communicate with each otherwithin a home network. User equipment devices can communicate with eachother directly via short-range point-to-point communication schemesdescribe above, via indirect paths through a hub or other similar deviceprovided on a home network, or via communications network 414. Each ofthe multiple individuals in a single home may operate different userequipment devices on the home network. As a result, it may be desirablefor various media guidance information or settings to be communicatedbetween the different user equipment devices. For example, it may bedesirable for users to maintain consistent media guidance applicationsettings on different user equipment devices within a home network, asdescribed in greater detail in Ellis et al., U.S. patent applicationSer. No. 11/179,410, filed Jul. 11, 2005. Different types of userequipment devices in a home network may also communicate with each otherto transmit content. For example, a user may transmit content from usercomputer equipment to a portable video player or portable music player.

In a second approach, users may have multiple types of user equipment bywhich they access content and obtain media guidance. For example, someusers may have home networks that are accessed by in-home and mobiledevices. Users may control in-home devices via a media guidanceapplication implemented on a remote device. For example, users mayaccess an online media guidance application on a website via a personalcomputer at their office, or a mobile device such as a PDA orweb-enabled mobile telephone. The user may set various settings (e.g.,recordings, reminders, or other settings) on the online guidanceapplication to control the user's in-home equipment. The online guidemay control the user's equipment directly, or by communicating with amedia guidance application on the user's in-home equipment. Varioussystems and methods for user equipment devices communicating, where theuser equipment devices are in locations remote from each other, isdiscussed in, for example, Ellis et al., U.S. Pat. No. 8,046,801, issuedOct. 25, 2011, which is hereby incorporated by reference herein in itsentirety.

In a third approach, users of user equipment devices inside and outsidea home can use their media guidance application to communicate directlywith content source 416 to access content. Specifically, within a home,users of user television equipment 402 and user computer equipment 404may access the media guidance application to navigate among and locatedesirable content. Users may also access the media guidance applicationoutside of the home using wireless user communications devices 406 tonavigate among and locate desirable content.

In a fourth approach, user equipment devices may operate in a cloudcomputing environment to access cloud services. In a cloud computingenvironment, various types of computing services for content sharing,storage or distribution (e.g., video sharing sites or social networkingsites) are provided by a collection of network-accessible computing andstorage resources, referred to as “the cloud.” For example, the cloudcan include a collection of server computing devices, which may belocated centrally or at distributed locations, that provide cloud-basedservices to various types of users and devices connected via a networksuch as the Internet via communications network 414. These cloudresources may include one or more content sources 416 and one or moremedia guidance data sources 418. In addition or in the alternative, theremote computing sites may include other user equipment devices, such asuser television equipment 402, user computer equipment 404, and wirelessuser communications device 406. For example, the other user equipmentdevices may provide access to a stored copy of a video or a streamedvideo. In such embodiments, user equipment devices may operate in apeer-to-peer manner without communicating with a central server.

The cloud provides access to services, such as content storage, contentsharing, or social networking services, among other examples, as well asaccess to any content described above, for user equipment devices.Services can be provided in the cloud through cloud computing serviceproviders, or through other providers of online services. For example,the cloud-based services can include a content storage service, acontent sharing site, a social networking site, or other services viawhich user-sourced content is distributed for viewing by others onconnected devices. These cloud-based services may allow a user equipmentdevice to store content to the cloud and to receive content from thecloud rather than storing content locally and accessing locally-storedcontent.

A user may use various content capture devices, such as camcorders,digital cameras with video mode, audio recorders, mobile phones, andhandheld computing devices, to record content. The user can uploadcontent to a content storage service on the cloud either directly, forexample, from user computer equipment 404 or wireless usercommunications device 406 having content capture feature. Alternatively,the user can first transfer the content to a user equipment device, suchas user computer equipment 404. The user equipment device storing thecontent uploads the content to the cloud using a data transmissionservice on communications network 414. In some embodiments, the userequipment device itself is a cloud resource, and other user equipmentdevices can access the content directly from the user equipment deviceon which the user stored the content.

Cloud resources may be accessed by a user equipment device using, forexample, a web browser, a media guidance application, a desktopapplication, a mobile application, and/or any combination of accessapplications or the same. The user equipment device may be a cloudclient that relies on cloud computing for application delivery, or theuser equipment device may have some functionality without access tocloud resources. For example, some applications running on the userequipment device may be cloud applications, i.e., applications deliveredas a service over the Internet, while other applications may be storedand run on the user equipment device. In some embodiments, a user devicemay receive content from multiple cloud resources simultaneously. Forexample, a user device can stream audio from one cloud resource whiledownloading content from a second cloud resource. Or, a user device candownload content from multiple cloud resources for more efficientdownloading. In some embodiments, user equipment devices can use cloudresources for processing operations such as the processing operationsperformed by processing circuitry described in relation to FIG. 3.

A supplemental content asset may be any content that may be accessed bya secondary device to accompany a primary content asset that has beenaccessed by a primary user device, and is related to the primary contentasset. The supplemental content asset may be any suitable content thathas some relation to a primary content asset being provided on theprimary user device. For example, the primary content asset and thesupplemental content asset may be related based on similar subjectmatter, tempo, or any other suitable relationship. Further descriptionsof possible relationships are provided below.

Providing a supplemental content asset may be any action performed byeither a primary user device accessing a primary content asset, asecondary device accessing a supplemental content asset related to theprimary content asset, or both the primary user device and the secondarydevice. The term “supplementing content assets”, “supplementing ofcontent assets”, or other similar terms may be used in lieu of“providing a supplemental content asset”. The providing of supplementalcontent asset may be performed by an application running on a userdevice. In some embodiments, the application may be an interactive mediaguidance application. In some embodiments, the application may be awidget. Depending on whether a user device is accessing a content assetthat may be supplemented by another device, or is accessing asupplemental content asset to accompany a content asset accessed onanother device, the application or different applications may be used.

Metadata is descriptive information that may be contained in the headerfor a content file. Metadata may include title, genre, duration,category, file type, file format, timing information, featureinformation, or any other suitable information. The timing informationof metadata may include lengths of a content asset, such as length of asong or video. The metadata may vary depending on the type of content.For example, image files may contain information about color maps andresolution of the images. Audio files may include information about thetype of audio encoding. Video files may include information about thetype of video encoding and indices within the file corresponding todifferent scenes.

FIG. 5 illustrates examples of user devices and electronic devices,discussed previously in reference to FIG. 3 and FIG. 4, that may provideprimary and supplemental content assets in accordance with someembodiments of the present disclosure. The user devices may includeset-top box 510, television equipment 520, stereo device 530, speakers540, gaming console 550, laptop computer 560, personal digital assistant(PDA) 570, or any other suitable user devices or equipment. In someembodiments, a user device may act as a media center that stores contentfor access. For example, a laptop computer may include a library ofcontent files stored in an internal hard drive.

Additional media files may be stored into the laptop computer. Contentfiles may be played back on the laptop computer, or accessed from thelaptop by another user device for play back on the other user device. Insome embodiments, a primary user device may access content from asecondary device. For example, a television may display content that isstreamed from a set-top box.

Any combination of these devices may be used in a system that provides asupplemental content asset. For example, a system may include a gamingconsole, a television and a stereo. A user may play a video game on agaming console that may be displayed on the television, and may besupplemented by the stereo. In response to accessing the video game, thegaming console may transmit information to the stereo about the accessedvideo game. The stereo may search a local content library on the stereofor a music file that is relevant to the video game and play back therelevant music file to accompany the video game during game play by theuser.

A primary user device may be any user device, as described above inreference to FIG. 4 and FIG. 5, that accesses a content asset that maybe supplemented by a supplemental content asset accessed by another userdevice. A secondary device may be any electronic device, as describedabove in reference to FIG. 4 and FIG. 5, that accesses a supplementalcontent asset to supplement a primary content asset accessed from aprimary user device. For example, a user may access a primary contentasset from a primary user device. In response to accessing the primarycontent asset, the primary user device and a secondary device maycommunicate to determine what supplemental content asset to provide atthe secondary device to supplement the primary content asset. In someembodiments, a secondary device may be an electronic device that doesnot have a user input. For example, a media center may be accessiblethrough network connections, but not directly through a keyboard ormouse.

In some embodiments, a user device may operate as a primary user deviceor a secondary device, depending on usage. For example, a gaming consolemay act as a primary user device when a user is playing a video gamethat is to be supplemented by a supplemental content asset, or as asecondary device when the gaming console may be used as a media centerto provide a supplemental content asset to another primary user device.As an example, a laptop may act as a primary user device when used toplay back music videos, or may act as a secondary device to play a musictrack as a supplemental content asset to a video game being played on aprimary gaming console device. As an example, a PDA may act as a primaryuser device when a user accesses music or videos on the PDA, or may actas a secondary device when providing web sites as a supplemental contentasset to a documentary video being watched on a primary laptop computer.

In some embodiments, a primary user device may communicate with a singlesecondary device, herein referred to as a nexus, that coordinatesfunctions of one or more secondary devices to provide a supplementalcontent asset. The nexus device may improve efficiency of a primary userdevice when multiple secondary devices provide a supplemental contentasset. By communicating with a nexus device, the number of operationsperformed on a primary user device may be reduced. For example, insteadof communicating with five secondary devices with each providing asupplemental content asset, a primary user device may insteadcommunicate with the secondary nexus device, which communicates with thefive secondary devices.

For example, set-top box 510 may act as a secondary nexus device thatcommunicates with secondary television equipment 520 and secondarystereo equipment 530 to present a supplemental content asset tosupplement a primary content asset accessed at a primary user device,such as a laptop computer 560. When used to access a video game, laptopcomputer 560 may transmit information about the accessed video game toset-top box 510. In response to receiving the information about theaccessed video game, set-top box 510 may communicate with stereo device530 to search for a music file, such as a soundtrack, that is relevantto the accessed video game to play as supplemental content asset.Set-top box 510 may also communicate with television equipment 520, inaddition to stereo 530, to search for a video file or slideshow from aninternal media server of the television equipment 520 that is relevantto the accessed video game to display on the television equipment 520 asa supplemental content asset. The set-top box 510 may communicateinformation about the video game received from the laptop computer 560to the television equipment 520 and the stereo equipment 530 tofacilitate the search for relevant content. Set-top box 510 may alsocommunicate with a remote server (not shown) to search for anadvertisement relevant to the accessed video game to display ontelevision 520 as a supplemental content asset to the video gameaccessed on laptop 560.

In some embodiments, a primary user device may communicate independentlywith several secondary devices, instead of through a secondary nexusdevice. For example, television equipment 520 and stereo equipment 530may act as secondary devices to a laptop computer 560 that may act as aprimary user device. In response to receiving a user input to access avideo game, laptop computer 560 may communicate directly with televisionequipment 520 and stereo equipment 530, instead of through set-top box510, to provide a supplemental content asset to the accessed video game.The laptop computer 560 may transmit information about the video game totelevision equipment 520 and stereo equipment 530. In response toreceiving the information, television equipment 520 may search aninternal local media server for a video file that is relevant to thevideo game accessed by the laptop computer. In response to receiving theinformation about the video game, stereo equipment 530 may search acontent library on the stereo for a song related to the video game.Television equipment 520 may provide the relevant video file and stereoequipment 530 may provide the relevant song as a supplemental contentasset. Television equipment 520 and stereo equipment 530 are able toprovide supplemental content asset without a secondary nexus device suchas set-top box 510.

In some embodiments, presentation of supplemental content may beperformed in response to detection of a primary user device that hasplayed back a portion of a primary content asset within a proximity ofthe secondary device, or is in the process of playing back a primarycontent asset. For example, a user may walk near a secondary device witha primary device. If the primary device has played back a primarycontent asset, the processing circuitry 306 of the primary device maytransmit playback information that includes a status that indicates theprimary content asset is paused, and the elapsed playback time of theprimary content asset. Detecting circuitry 307 of a secondary device maydetect the primary device and processing circuitry 306 of the secondarydevice may receive playback information from the primary device.Processing circuitry 306 of the secondary device may offset the startingtime of the supplemental content. When playback of the primary contentasset on the primary device is resumed, the processing circuitry 306 ofthe secondary device may playback the supplemental content asset fromthe offset starting time to match the primary content asset. If theprimary device is currently playing back the primary content asset,processing circuitry of the secondary device may offset the startingtime of the supplemental content asset and start playback immediately.Processing circuitry of the primary device may transmit playbackinformation that indicates that the status is playing, and an elapsedtime of playback up to the transmission of the playback information.Processing circuitry of the secondary device may offset the startingtime of the supplemental content asset and start playback of thesupplemental content asset immediately to match the playback of theprimary content asset on the primary device.

FIG. 6 illustrates a flowchart of steps that describes a process forsupplementing a primary content asset accessed on a user device inaccordance with some embodiments of the present disclosure. At step 610,a user accesses content using a primary user device. For example, a usermay play a video game on a gaming console device 550. In response toaccessing the video game, the gaming console device 550 may generate acontent information message for transmission to a stereo 530 or othersuitable secondary device. The generated message may contain informationabout the accessed video game that is used by the stereo 530 or othersecondary device to search for relevant content. The format and contentof the message is described in further detail in reference to FIG. 15.

For example, processing circuitry 306 of the primary user device mayreceive a user input from a user input interface 310 to access a contentasset, such as the video game. In response to receiving the user input,processing circuitry 306 of the primary user device may generate amessage containing information about the accessed primary content asset.

At step 620, the primary user device transmits the message to asecondary device. For example, the gaming console, or another primaryuser device, may transmit a message to the stereo, or another secondarydevice, that contains information about the video game such as genre ortitle. In this example, the genre of video game may be classified asaction. The stereo may receive and process the message to extractinformation about the video game. For example, at the primary userdevice, processing circuitry may communicate the message on an I/O path,across communications network 414 to the secondary device. At thesecondary device, processing circuitry may receive the message from anI/O path. Processing circuitry of the secondary device may extractinformation from the message about the primary content asset accessed atthe primary user device. The processing circuitry and I/O paths of theprimary and secondary devices may be substantially similar to theprocessing circuitry 306 and I/O path 302 shown in FIG. 3.

At step 630, the secondary device searches a database for a supplementalcontent asset that is relevant to the primary content asset, based onthe information about the primary content asset. In some embodiments,the database may include content listings, where each content listing isassociated with a content file stored in a library of content files. Forexample, as described above in reference to FIG. 3, storage 308 of asecondary device may store a library of content files and a database ofcontent listings. Each content listing may be associated with a contentasset file of the stored library or a content asset accessible to thesecondary device, such as web content asset. In some embodiments, thedatabase may include content files. For example, storage 308 of asecondary device may store a database of content files. Relevancy of thepotential supplemental content asset may be determined by comparingmetadata, feature information, mood information and/or any othersuitable information of the primary content asset and correspondinginformation of potential supplemental content assets in the database.For example, a stereo acting as a secondary device may search for audiofiles that are relevant, based on title or genre, to a video game beingplayed on a gaming console acting as a primary user device. The stereo530 may search for stored content that is relevant to the video game, byusing information extracted from the transmitted message. If the videogame is classified as the action genre, the stereo may search for audiofiles from an internal database that are relevant to the action genreand return results that include soundtracks from action movies andaction television shows.

In some embodiments, the secondary device may search for relevantcontent based on features extracted from the primary content asset. Forexample, if the primary content asset is a video file, the stereo maysearch for relevant content by video features or image features. Forexample, the color palette of the video may be extracted. Processingcircuitry 306 of a secondary device may store a lookup table in storage308 that correlates different schemes of the color palette with metadataterms. For example, processing circuitry 306 may store an entry in thelookup table that correlates grayscale color palettes to music from the1950s time era. If the secondary device is a stereo, processingcircuitry 306 may determine that the color palette of the primary videocontent is grayscale, and in response search the lookup table and returnthe metadata terms for “1950s music.” Processing circuitry 306 of thestereo may then search for music files with metadata associated with the1950s time era when black and white televisions were used. Relevancymetrics will be further discussed later in the specification inreference to FIG. 8.

In some embodiments, the relevant content may be accessed from a localstorage on the secondary device. For example, at the secondary device,processing circuitry 306 may access storage 308 to search for contentthat is relevant to the primary content asset accessed on the primaryuser device.

In some embodiments, the relevant content may be accessed from a remotestorage. For example, at a secondary device, processing circuitry 306may transmit a command along I/O path 302 through communications network414 to content source 416 and media guidance data source 418 to searchfor content that is relevant to the primary content asset accessed onthe primary user device. The search methods will be described in furtherdetail in reference to FIG. 8. In some embodiments, the results of thesearch for relevant content may be accessed from a combination of localstorage and remote storage.

At step 640, the secondary device selects a relevant supplementalcontent asset to access based on the search results that were determinedin step 630. For example, the processing circuitry 306 of the stereo, oranother secondary device, may select an action movie sound track fromthe search results including sound tracks. In some embodiments,processing circuitry 306 of a secondary device may select a contentasset with the highest relevancy metric from among the relevant contentdetermined in step 630 as a supplemental content asset.

At step 650, the secondary device presents the relevant content on asecondary device as a supplemental content asset to the primary contentasset. For example, the stereo 530 may play back the action moviesoundtrack selected in step 640, to accompany the video game during gameplay by the user. For example, at a secondary device, processingcircuitry 306 may output the selected relevant content to display device312 and speakers 314. In some embodiments, the relevant content maycontain only a video component and no audio component. In such cases,processing circuitry 306 may output the video content to display 312. Insome embodiments, the relevant content may only contain an audio ormusic component, and no video component. Processing circuitry 306 mayoutput the audio or music component to speakers 314.

In some embodiments, the secondary device may present the relevantsupplemental content asset on the same secondary device. As an example,the secondary device may be stereo 530. Steps 620, 630, 640, and 650could be performed by the secondary stereo 530, communicating directlywith a primary gaming console. In some embodiments, the secondary devicemay present the relevant content on a different secondary device, forexample when a nexus secondary device communicates with anothersecondary device. In some embodiments, a secondary device that providescommands to present content may be a master secondary device, and asecondary device that receives the commands and presents the content maybe a slave secondary device. For example, a system may include a gamingconsole, set-top box, and a stereo. The gaming console may act as aprimary user device. A user may access an action video game at thegaming console.

In response, the gaming console may transmit a generated message to theset-top box, containing information about the accessed video game. Theset-top box may then process the received message to extract informationabout the accessed primary content asset, and select a relevant content,such as an audio soundtrack, based on the extracted information. Oncethe set-top box has selected a content asset, it may direct a stereo toplayback the selected audio soundtrack. A series of actions may beperformed by the secondary device, during an event of presenting thesupplemental content asset. For example, at secondary set-top box 510,processing circuitry 306 may access a file corresponding to an actionmovie soundtrack that is relevant to the primary content asset, theaction video game. The set-top box may then transmit the relevantsoundtrack and a playback command along I/O path 302 acrosscommunications network 414 to another secondary device such as stereo530. At secondary stereo 530, processing circuitry 306 may receive therelevant content and play back command from I/O path 302 and play backthe received content.

FIG. 7 illustrates a flowchart of steps that describes a process forsupplementing a primary content asset accessed on a primary user devicein accordance with some embodiments of the disclosure. The processillustrated by FIG. 7 may be carried out on a user device that is actingas a primary user device or a secondary device. For example, a portablegaming device with wireless capabilities may scan for surroundingsecondary devices that may provide a supplemental content asset.Depending on the type of secondary devices detected, the portable gamingdevice may generate a message template. The portable gaming device mayuse the message template to generate content information messages thatinclude information about a primary content asset accessed by theportable gaming device, and type of supplemental content asset that isto be provided by the secondary devices. For example, if only audiodevices such as stereos are detected, the message template may indicatethat only audio content should be presented. The message template mayalso be generated based on input from a user of the primary user device.For example, in response to a user input indication that a supplementalvideo content asset is desired, processing circuitry of a user devicemay generate a message template that indicates supplemental videocontent assets are to be searched and provided by the detected secondarydevices. In response to a user indication that a supplemental audiocontent asset is desired, a message template may be generated thatindicates a supplemental audio content asset is desired. In someimplementations, the user may specify that only one type of supplementalcontent asset is desired. For example, in response to a user inputindication that only supplemental video content are desired, processingcircuitry of a primary user device may generate a message template thatindicates only supplemental video content are to be searched andprovided by the detected secondary devices. The supplemental contentasset type preference indicated in the message template may reduce thetime used by processing circuitry of a secondary device to search for asupplemental content asset.

After detecting secondary devices and generating a message template, theportable gaming device may select one or more secondary devices toprovide a supplemental content asset. As an example, the portable gamingdevice may select a stereo 530 as a secondary device. The portablegaming device may generate and transmit a content information message tothe stereo device. The portable gaming device may then poll for afeedback information message from stereo 530 that indicates whether arelevant supplemental content asset has been found. In some embodiments,the feedback information message may also include a timestamp andtimeline with markers to coordinate presentation of the primary contentasset and supplemental content asset.

In response to receiving the feedback information message, the portablegaming device may generate a synchronization information message totransmit to the stereo device. The synchronization information messagemay serve as an acknowledgment by the portable gaming device, andprovide a timestamp and timeline to the stereo device to coordinatepresentation of the primary content asset and the supplemental contentasset.

As an example, a timeline received in the feedback information messagefrom the stereo device may contain markers that correspond to a playlistof relevant supplemental content assets on the stereo device. A timelinesent in a synchronization information message from the portable gamingdevice to the stereo device may include adjustments to the markerpositions received in the feedback information message to indicate whenthe stereo should play back certain songs, to coordinate with game playof a video game on the portable gaming device.

Steps 710, 720, and 730 can be considered as a set-up phase of theprocess for the primary user device. A set of steps corresponding to asimilar set-up phase will be discussed in reference to FIG. 8. Steps710, 720, and 730 may be performed at any suitable time. For example,before a primary user device and a secondary device communicateinformation about a primary content asset that is accessed on theprimary user device. In some embodiments, the steps may be performedbefore a primary content asset is accessed at a primary user device.

For example, a gaming console may first detect available secondarydevices that are physically connected by cables or within a range ofwireless communication. Executing the steps of a set-up phase before aprimary user device accesses a primary content asset and before asecondary device accesses supplemental content asset allows the primaryuser device and secondary device to perform initialization operationsthat may require significant overhead in run time. Running the set-upsteps may reduce the time needed to provide a supplemental content assetat a later time because of the elimination of the initial overhead runtime.

Initialization operations may be operations performed at least once uponpower-up of a user device, or connection of a user device to anotheruser device, that may be included in the initial overhead. In someembodiments, the steps may be performed upon power-up of the possibleprimary user device. In some embodiments, the steps may be performedperiodically as new secondary devices may enter the system.Initialization operations may include indexing stored content to enablefast searching and access.

At step 710, a primary user device determines if any possible secondarydevices have been detected in the surrounding vicinity. For example, awireless portable gaming device may scan for secondary wireless devicesin the vicinity. A portable gaming device may also detect secondarydevices that are physically connected, for example, by a USB cable. Insome embodiments, the primary user device may initiate a scan forsecondary devices by actively transmitting a message to secondarydevices. For example, at a primary user device, processing circuitry 306may transmit messages on I/O path 302 across communications network 414to a secondary device. In some implementations, the secondary device andprimary user device may communicate across a wireless network. Forexample, the transmitted message may be a beacon broadcast to all activesecondary devices. The transmitted message may also be a probe requestdirected toward a secondary device.

In some embodiments, the primary user device may passively scan formessages from secondary devices. For example, at the primary userdevice, processing circuitry 306 may receive beacon or probe requestmessages from I/O path 302. If a secondary device is detected, theprocess proceeds to step 720, otherwise the process repeats at step 710.In some embodiments, wireless detection techniques as described inreference to FIG. 4 are used to detect secondary devices.

At step 720, the primary user device has detected at least one secondarydevice by receiving from the secondary device a message, such as abeacon or probe response. The primary user device receives aninformation message from a secondary device, and determines informationabout the detected secondary device. In some implementations, the deviceinformation message may be received as part of the detecting step 710.In some implementations, the device information message may be receivedafter the detecting step 710. For example, a portable gaming device mayhave received beacon messages from secondary devices in the surroundingarea, and may process the beacon messages to extract information aboutthe secondary devices in the surrounding area. The device informationmessage may contain capabilities information such as the type of contentthat the secondary device is able to present (e.g., audio, video,Internet content, multimedia etc.), type of information such as whetherthe secondary device is a nexus, whether the secondary device contains acontent library of stored files, history information such as whether thesecondary device has previously provided a supplemental content asset tothe primary user device, or any other suitable information related tothe providing of a supplemental content asset.

Processing circuitry of the primary user device may extract informationabout the capabilities of the secondary device from a device informationmessage received from the secondary device. As a result of processingreceived device information messages, the primary user device maydetermine that there is a secondary stereo 530, a secondary television520 and a secondary set-top box 510. The primary user device maydetermine that the secondary stereo 530 has a library of digital musicfiles, and that the secondary set-top box 510 has a library of digitalvideo files and Internet content. For example, at the primary userdevice, processing circuitry 306 may receive from I/O path 302 a deviceinformation message transmitted from a secondary device. Processingcircuitry 306 may then extract information about the secondary devicefrom the received device information message and store information aboutthe secondary device in storage 308.

In some embodiments, the primary user device may determine whichdetected secondary device may be used to provide a supplemental contentasset based on the capabilities information. For example, processingcircuitry 306 of the primary user device may receive user input from aconfiguration screen 1100 of FIG. 11, indicating what type of contentshould be provided as a supplemental content asset. Processing circuitry306 may store configuration information indicating the type ofsupplemental content asset that should be provided. Processing circuitry306 may select a secondary device to provide a supplemental contentasset based on the capabilities of the secondary device and the storedconfiguration information.

At step 730, based on information about detected secondary devicesdetermined from step 720, the primary user device generates a messagetemplate that is used to communicate information about a primary contentasset that is accessed on the primary user device, and information abouta requested type of supplemental content asset. This message templatemay be used by a secondary device to select content that is relevant tothe primary content asset. For example, the primary user devicedetermines that only a secondary stereo device 530 is available, it maygenerate a message template that contains acoustic feature information,but not visual feature information. The selective inclusion ofinformation reduces the size of the message that may be communicatedbetween a primary and secondary device. The smaller message size reducesthe communication time of the content information message.

As an example, processing circuitry 306 may access information aboutdetected secondary devices from storage 308. If processing circuitry 306determines from the information that only a secondary stereo device hasbeen detected, it may generate a message template that includes fieldsfor metadata information, and fields for acoustic feature informationabout a content asset. If processing circuitry 306 determines that asecondary television device has been detected, it may generate a messagetemplate that includes fields for metadata information about a contentasset, fields for acoustic feature information about the audio componentof a content asset, and visual feature information about the videocomponent of a content asset. In some embodiments, a message templatemay be generated for each type of secondary device that is detected. Forexample, if a stereo and television are detected, a first messagetemplate may be generated for the stereo and a second message templatemay be generated for the television. Processing circuitry 306 may storea generated message template in storage 308.

In some embodiments, processing circuitry 306 of the primary user devicemay generate the message template in part based on user profileinformation stored on the primary user device. For example, a user ofthe primary user device may specify a preference for audio instead ofvideo as a supplemental content asset from a secondary device. Userprofile information may resolve conflicts when selecting the type ofsupplemental content asset. For example, processing circuitry of aprimary laptop computer may detect a secondary television equipment 520and a secondary stereo equipment 530. If the user profile indicates apreference for video as a supplemental content asset, processingcircuitry of the primary laptop computer may generate a message templatethat includes fields for metadata that indicate only video contentshould be searched on a secondary device and fields for visual featureinformation that may be used to determine relevancy of a video. Step 730may be considered the last step of the set-up phase.

At step 740, the primary user device determines whether a user hasrequested to access a primary content asset on the primary user device.For example, a gaming console may determine whether a user has tried toplay a video game. Processing circuitry 306 may receive a user inputfrom user input interface 310 indicating a desire to play a video game.If the processing circuitry of the primary user device receives a userrequest to access a primary content asset, it proceeds to step 750.Otherwise, the primary user device will loop on step 740.

At step 750, the primary user device generates a content informationmessage containing information about the requested a primary contentasset from step 740. The content information message may be generatedusing the message template created in step 730. For example, in responseto receiving a user indication to play a game, a gaming console mayaccess the video game and extract information to generate a contentinformation message which includes metadata information such as titleand genre, and acoustic feature information from the soundtracks of thevideo games. In some embodiments, the feature information may begenerated from a specific scene or segment of the video game that isbeing accessed. For example, the acoustic feature information may beextracted from a soundtrack of a level of the video game that isselected for game play. Processing circuitry 306 may access storage 308to retrieve the generated message template and the requested content.Processing circuitry 306 may then generate a message based on themessage template and the requested content.

At step 760, the primary user device transmits a content informationmessage generated in step 750 to the secondary device. For example, agaming console may transmit a generated content information message to astereo, set-top box, television, or other suitable secondary device.Processing circuitry 306 of the primary user device may transmit thegenerated information message on I/O path 302 across communicationsnetwork 414 to a secondary device.

At step 770, processing circuitry of the primary user device determineswhether a feedback information message has been received from thesecondary device. Feedback information may be used to coordinate thepresentation of a supplemental content asset with the presentation of aprimary content asset accessed on the primary user device. For example,when a user plays a video game, it may be desirable to play back asupplemental soundtrack from a secondary stereo that is synchronizedwith the playback of the video scenes from the primary video game.

In some embodiments, the feedback information message may include asimple indicator of whether the secondary device was successful orunsuccessful in finding a relevant supplemental content asset. In someembodiments, the feedback information message may contain information tocoordinate playback of the supplemental content asset and primarycontent asset. For example, a feedback information message from asecondary stereo may include a timestamp and timeline. The timestamp mayindicate to the primary user device when to start playing the primarycontent asset. The secondary device may also play back content startingat the same timestamp. The timeline may include markers to indicate tothe primary user device when the supplemental content asset may end. Atthese markers, the primary user device may pause playback of the primarycontent asset, or may send another media information message to thesecondary device to request selection of a new supplemental contentasset. For example, processing circuitry 306 of a primary user devicemay receive a feedback information message from a secondary deviceacross communications network 414 on I/O path 302. If the primary userdevice receives feedback information from the secondary device, theprocess proceeds to step 780. Otherwise, the process loops on step 770.

At step 780, in response to receiving the feedback information messagefrom the secondary device, the primary user device generates andtransmits a synchronization information message to the secondary device.Synchronization information is used to coordinate presentation of theprimary content asset accessed on the primary user device withpresentation of the supplemental content asset accessed on the secondarydevice. For example, similar to the description of a feedbackinformation message, a primary user device may transmit timeline andtimestamp information to the secondary device to signal when playback ofthe supplemental content asset should occur. At a primary user device,processing circuitry 306 may generate and transmit a synchronizationinformation message on I/O path 302 across communications network 414 toa secondary device. At a secondary device, processing circuitry 306 mayreceive a synchronization information message on I/O path 302 fromcommunications network 414. Processing circuitry 306 may process thereceived message to extract a timestamp and timeline including markers.Processing circuitry 306 then presents a supplemental content asset ondisplay 312 and speakers 314 as a function of the time extracted fromthe timestamp.

FIG. 8 illustrates a flowchart of steps that describes a process forsupplementing content by a possible secondary device in accordance withsome embodiments of the present disclosure. For example, a secondarydevice such as a stereo may provide soundtracks as a supplementalcontent asset to a video game that is being played on a gaming console.

Steps 810 and 820 can be considered as a set-up phase of the process forthe secondary device. During this phase, the secondary device maygenerate a database of content or content listings that can be searchedto find a supplemental content asset relevant to a primary contentasset. For example, processing circuitry of a stereo may generate adatabase of content or content listings for each music file storedwithin a local media server within the stereo. If the secondary deviceacts as a nexus device, it may detect other secondary devices duringthis phase. For example, a system of secondary devices may include aset-top box, a DVD player and a stereo. The set-top box may act as anexus device to coordinate the operation of the DVD player and stereo.If the set-top box acts as nexus, the set-top box first detects othersecondary devices such as the DVD player and the stereo. The devices maybe connected by physical connections such as USB cables, Ethernet cablesor any other suitable cables. The devices may be connected by wirelessconnections, as described previously in reference to FIG. 3 and FIG. 4.

In some embodiments, several secondary devices independently communicatewith a primary user device without the use of a secondary nexus deviceas an intermediary. In these embodiments, the secondary device mayinclude a local content source and a local media guidance data source.Processing circuitry 306 of the secondary device may generate a databaseof its stored content, using similar concepts as described below inreference to the embodiments employing a secondary nexus device.

In some embodiments, a nexus device is used, and the other detectedsecondary devices may collectively appear as a single secondary deviceto the primary user device. In some implementations, the secondary nexusdevice may act as a router that does not store any content and providesa supplemental content asset for a primary user device by coordinatingwith other secondary devices. In some implementations, the secondarynexus device may be capable of providing a supplemental content assetfor a primary user device without coordinating with other secondarydevices. In these implementations, the secondary nexus may internallystore content libraries, or access content libraries stored on a remoteserver, or on the Internet.

At step 810, the secondary device detects if there are any othersecondary devices available. For example, the secondary device may be anexus device, and may detect other secondary devices through physicalconnections or through wireless network connections as described inreference to FIG. 4. For example at the secondary nexus device,processing circuitry 306 may receive a device information message at I/Opath 302 from a secondary device indicating capabilities and one or morecontent libraries. This message may be similar to the messages receivedby a primary user device, such as a beacon or probe response, whenscanning for secondary devices as described in reference to step 710 ofFIG. 7. In some implementations, this message may be a deviceinformation message, a content information message, any other suitablemessage or any combination thereof.

At step 820, processing circuitry of a secondary device generates adatabase of content or content listings based on the available contentlibraries. For example, a system may include a set-top box that acts asa nexus, a DVD player and a stereo. A database generated by such asystem may include content or a content listing for the content that isstored on the stereo, stored in the discs loaded in the DVD player, ormay be available to the set-top box from a remote content source 416.The database listings for content may include metadata from the contentfile, and acoustic and visual feature information extracted from thecontent and timing information as described previously. In someimplementations, the metadata may include timing information, acousticfeature information, and/or visual feature information. In someembodiments, the secondary nexus device communicates with other slavesecondary devices to build an aggregate database of content listingsthat may be used to select a relevant supplemental content asset. Forexample, at a slave secondary device, processing circuitry may transmita message on an I/O path across communications network 414 to thesecondary nexus device providing information about its content library.At the secondary nexus device, processing circuitry may receive amessage on an I/O path containing information about the content libraryof the slave secondary device. The processing circuitry of the nexusdevice may then compile information about the content libraries of otherslave secondary devices. The processing circuitry and I/O paths of theslave secondary devices and secondary nexus devices may be substantiallysimilar to processing circuitry 306 and I/O path 302 of the deviceillustrated in FIG. 3.

To assist in the searching for relevant content and correlation ofcontent files, features may be extracted by analysis of content files.For example, acoustic features may be extracted from audio files thatserve as metrics for measuring bass, treble, rhythm, pitch distributionor any other suitable acoustic characteristics. Examples of acousticfeatures include mel-frequency cepstral coefficients (MFCCs), cepstralcoefficients, formants and frequency. As an example, a user may accessan audio file that has a large bass component. This may be observed by alarge weighting of a low frequency coefficient extracted from a Fourierspectrum of an audio file. The relevancy of content may be determined bycomparing these acoustic features. Similarly, visual features includingbrightness, contrast, color range, edge count, corner count, or anyother suitable visual feature may be extracted from image files andvideo files. As with acoustic features, the relevancy of content may bedetermined by comparing visual features.

In some embodiments, steps 810 and 820 may be skipped. For example, if asecondary device is not a nexus device, it may not detect othersecondary devices.

At step 830, the secondary device detects if a primary user device isavailable. For example, detecting circuitry 307 may use wirelessdetection techniques to actively or passively scan for a primary userdevice. In an active scan, detecting circuitry 307 may broadcast probemessages on I/O path 302 to query the presence of other wireless primaryuser devices. If there is a wireless primary user device present,detecting circuitry 307 of the secondary device may receive proberesponse messages from I/O path 302 identifying the primary user device.In a passive scan, detecting circuitry 307 may poll for beacon messagesbroadcast by a primary user device that identifies the primary userdevice.

In some embodiments, the probe response or beacon messages received fromthe primary user device may include playback information that describesa primary content asset that may be playing on the primary user device.Processing circuitry 306 of the secondary device may extract playbackinformation for later use in the presentation of supplemental content.If a primary device is detected, the process proceeds to step 840,otherwise the process loops on step 830.

At step 840, the secondary device polls for a content informationmessage from a primary user device that provides information about aprimary content asset that has been accessed on the primary user device.For example, a secondary stereo may poll for a content informationmessage from a gaming console that provides information on a video gamebeing played. The content information message may be used to search fora supplemental content asset from the stereo to supplement the videogame. At the secondary device, processing circuitry 306 may receive theinformation message from I/O path 302. In response to receiving themessage, processing circuitry 306 may extract relevant information fromthe content information message. In some embodiments, the contentinformation message may include playback information as described above.For example, the playback information described above may be included inthe content information message instead of the beacon or probe responsemessages.

In some embodiments, steps 830 and 840 may be combined into one step.For example, instead of two steps of receiving a probe response orbeacon frames, and receiving a content information message, processingcircuitry 306 of a secondary device may receive a content informationmessage that indicates the presence of a primary user device. Theplayback information may be included in the content information message.The use of separate steps for 830 and 840, or a combined step for 830and 840, may depend on design considerations. For example, beacon, probeand probe response messages may be short communications that requireminimum bandwidth for transmission, while content information messagesmay be large communications that require longer transmission time. Byusing constant transmission and reception of small beacon, probe andprobe response messages to determine presence of a primary device,instead of constant transmission and reception of large contentinformation messages, bandwidth in the air may be more efficientlyallocated. Alternatively, periodically transmitting a contentinformation message over the air may reduce programming complexity.

If the content information message has been received from the primaryuser device, the process will proceed to step 850. Otherwise the processwill loop at step 840.

At step 850, processing circuitry of the secondary device receiving thecontent information message from the primary user device searches adatabase compiled above in step 820 for possible supplemental contentassets that is relevant to the primary content asset accessed on theprimary user device. For example, a secondary stereo may search acontent library of sounds for audio soundtracks to play back asaccompaniment to a video game being accessed on a primary gamingconsole. Relevancy of content may be determined using informationprocessed from the content information message in step 840. For example,metadata, acoustic features, or visual features may be compared andcorrelated. Comparing similar or identical content types may bestraightforward as many commonly shared fields may exist in the metadataand acoustic and visual features. For example, a genre and keywords fromtitles contained in the metadata may be compared to determine relevancyof two mp3 audio files. For example, extracted frequency coefficientfeatures may be compared to determine relevancy of two mp3 audio files.

To determine relevancy of two files having different file types,algorithms may be used to determine how to compare the different filetypes. For example, the metadata fields of the two different files maybe compared to determine common fields that may be used for comparison.The acoustic and visual features of the two different files may becompared to determine common fields that may be used for comparison.

In some embodiments, pre-set algorithms may be used to compare differentfile types to determine relevancy. In some embodiments, algorithms mayadaptively learn to correlate and categorize different file types basedon user access. As an example, processing circuitry 306 may extractmetadata and feature information from the message received in step 840and search a database of content assets and content listings stored instorage 308. Processing circuitry 306 may compute a relevancy metricbetween the content assets and content listings of the database and theprimary content asset accessed on the primary user device. The metricmay be a weighted average of several comparison metrics calculated bycomparing metadata and features between the two files. The comparisonmetrics may include Euclidean distance or ratios of metadata andfeatures.

In some embodiments, files of different media types may be correlatedand compared. For example, processing circuitry of a secondary devicemay search for relevant music to supplement a video that does not have asoundtrack. If there are no common acoustic or visual features betweenthe two different file types, different types of features may becorrelated. For example, music having a high bass component and slowrhythm is typically associated with melancholy emotions. Similarly,images having dark or gray color backgrounds are typically associatedwith melancholy emotions. To correlate the audio file and video file, afeature vector of frequency coefficients of the audio file may becompared against a feature vector of color indices of the video file. Afeature vector may be a single or multi-dimensional array of numericalelements corresponding to a specific metric. The feature vector offrequency coefficients may describe the relative proportion of differentfrequency bands that may occur in an audio file, and may be ordered in aone-dimensional array from low frequency to high frequency. For example,the frequency coefficients may correspond to different portions of amel-scale of frequency. The feature vector of color indices may describethe relative proportion of different colors that appear in a video. Inthe examples of frequency and color, a feature vector may be aone-dimensional array of numbers, in which each array element is anumber corresponding to a different frequency band or color index.

The feature vectors may be pre-processed before comparison by processingcircuitry of a secondary device. For example, if the feature vectors areof different length or dimensionality, processing circuitry 306 of asecondary device may resize the vectors to the same length and dimensionby combining numerical elements of the vector. The combination may bedone by calculating an arithmetic mean, geometric mean, or any othersuitable mathematical combination of the numerical elements. Afterresizing the feature vectors to similar sizes, the numerical data of thefeature vectors may be normalized. For example, each numerical elementof the feature vector may be scaled so that the sum of the elements inthe vector is equal to one.

Processing circuitry 306 of the secondary device may then analyze thefeature vectors using any suitable method of comparison. For example,processing circuitry 306 may calculate a dot product of the featurevectors to produce a result vector. The elements of the result vector,or any combination thereof may be a relevancy metric used to determinewhether the video file and audio file are relevant. For example,processing circuitry 306 may compare the elements of the result vectoragainst a threshold value, to determine whether the video file and theaudio file are correlated.

The threshold may be defined using any suitable approach. As referred toherein, a “threshold” may be a number or vector of numbers programmedinto an electronic device at time of manufacture, entered by a user, orcalculated by control circuitry of an electronic device. In someembodiments the threshold value may be varying. For example, controlcircuitry of an electronic device may calculate the threshold value mayas a fraction of a moving average of elements of previously determinedresult vectors. In some embodiments, the threshold may be calculatedbased on content assets that are indicated in user preferenceinformation as most commonly accessed by a user. Control circuitry maycalculate the threshold value as a weighted average of elements ofresult vectors calculated from comparing features of the most commonlyaccessed content assets. The threshold may be stored in storage 308, andretrieved by control circuitry 304 for use in comparisons.

At step 860, processing circuitry of a secondary device selects arelevant content for supplementing the primary content asset accessed onthe primary user device. For example, a secondary stereo may select amovie soundtrack as a supplemental content asset to accompany an actionvideo game accessed on a primary gaming console. In some embodiments,the secondary device may select from the database the content having thehighest relevancy metric. At a secondary device, processing circuitry306 may select the content from the database having the highestrelevancy metric computed in step 850. In some embodiments that use asecondary nexus device, the secondary nexus device may select the mostrelevant content for each slave secondary device. For example, a systemof secondary devices may include a nexus set-top box, a stereo and a DVDplayer. The set-top box may select two relevant supplemental contentassets: an audio file for playback on the stereo, and a DVD track forplayback on the DVD player. In some embodiments, the selecting of therelevant content may be performed by the processing circuitry of slavesecondary devices instead of the processing circuitry of the secondarynexus device. For example, the selection of the relevant audio file maybe performed by processing circuitry of the stereo, and the selection ofthe relevant DVD track may be performed by the processing circuitry ofthe DVD player.

For example, at the secondary nexus device, processing circuitry 306 mayselect a content asset having the highest relevancy metric from thelistings corresponding to each slave secondary device. Processingcircuitry 306 may then transmit a message on an I/O path 302 acrosscommunications network 414 to a slave secondary device indicating whichcontent is to be provided as a supplemental content asset.

In some embodiments, selection of the supplemental content asset by asecondary device may be based in part on user profile information. Forexample, relevancy of supplemental content asset may be affected by auser's age, gender or mood. User profile information may be used in thesearch for a relevant supplemental content asset. For example,processing circuitry 306 of a secondary device, such as a stereo, mayselect a different soundtrack based on whether the user of the primaryuser device is a male or female. Systems and methods for sharing mediausing social networks are discussed in greater detail in connection withKetkar, U.S. patent application Ser. No. 12/848,025, filed Jul. 30,2010, which is hereby incorporated by reference herein in its entirety.

The user profile information may be stored on a local storage device ona user device, or may be accessed from a remote server using acommunication network as described in reference to FIG. 4. In someembodiments, the user profile information may be communicated from aprimary user device to a secondary device in a content informationmessage. For example, processing circuitry 306 of a primary user devicemay transmit user profile information in a content information messageto a secondary device. In some embodiments, either a primary user deviceor secondary device may access the user profile information from aremote server or over the Internet. For example, the user profileinformation may be stored on a social network, including Facebook™,Twitter™ and MySpace™. In these social networks, users may post currentstatus and moods.

In some embodiments, the mood may be specifically selected by a user.For example, a user device may present a menu of a list of moods to auser. In response to receiving a selection of a mood from the user,processing circuitry 306 of the user device may connect to the user'saccount on a social network to post the mood information. A user devicemay then determine the mood of a user by retrieving the posted moodinformation from the social network. For example, processing circuitry306 of the user device may then connect to the user's account on asocial network to retrieve previously posted mood information.

In some embodiments, mood information may be extracted based on contentposted to a social network by a user. For example, a user may postcontent, such as a video, a link to a video, or text on a socialnetwork. Mood information may be extracted from this posted content. Insome implementations, the content may have mood information stored inmetadata. For example, a music video of a heavy metal genre may have ametadata field indicating that the mood corresponding to the video isangry. The mood metadata may be text, a number, set of numbers, anyother suitable metadata or any combination thereof. For example, themood metadata may be text information that describes a mood, such as“angry”, “happy”, “sad”, or “depressed”. The mood may be a number on ascale corresponding to different moods. For example, 1 may indicatehappy and 10 may indicate depressed. In such a scale, a number of 5 mayindicate a neutral mood.

The mood may be a series of numbers, corresponding to magnitude ofdifferent moods. For example, the mood information may include threenumbers, each on a scale of 0 to 10, corresponding to three separatemood categories such as happy, angry and depressed. The number on thescale indicates the extent of the mood. For example, a number 0 for thehappy category may indicate that a user is not happy, while a number 10for the happy category may indicate that a user is very happy.

The mood information in the metadata may be assigned when the contentfile is created, or may be modified at some time after creation of thefile. In some implementations, mood information may be pre-processed ordetermined by a server hosting the social network. For example,processing circuitry of a server may receive and analyze content postedby a user. For example, the processing circuitry may extract the moodmetadata information from a series of recently posted content (e.g., themost recent five posts, or any posts within a fixed time period in thepast, such as 10 hours). Using the extracted mood information from theposted content, the processing circuitry may calculate a mood of theuser by averaging the extracted mood information.

In some implementations, the mood information may be determined by auser device connecting to a social network. For example, processingcircuitry 306 of a user device may connect to the user's account on asocial network and retrieve the mood metadata information correspondingto content posted by the user.

In some implementations, a user device or server hosting the socialnetwork may determine mood information based on the content. Forexample, processing circuitry of the user device or the server mayanalyze keywords in the text of a message that is posted to the socialnetwork, or to be posted to the social network and determine a mood.

In some embodiments, a user device may post mood information to a user'saccount on a social network, based on mood information determined fromprimary content asset accessed by a user. For example, in response toreceiving a user request to access a primary content asset associatedwith an angry mood, such as a heavy metal album, processing circuitry306 of a primary user device may send an update to a server hosting asocial network account of the user, indicating that the user is angry.The mood information updates to the social network inform peers of theuser on the social network about the mood of the user.

In some embodiments, the social network may also use the moodinformation to enhance interaction between users of the social network.For example, processing circuitry 306 of a server hosting the socialnetwork may determine that the mood information for a first user hasbeen set to “depressed” for a predetermined threshold of time, such as aweek. In response, processing circuitry 306 may send a message to asecond user who is connected to the first user on the social network,informing that the first user is depressed and recommending that thesecond user communicate with the first user to affect the mood of thefirst user. The communication may be a text message, e-mail, voice mail,online greeting card, or any other suitable communication or combinationthereof.

As an example, processing circuitry 306 of a server hosting the socialnetwork may determine that the mood information for a first user haschanged to “angry.” In response, processing circuitry 306 may warn usersattempting to communicate with the first user that the first user isangry. For example, in response to receiving a request from a seconduser to send a message to a first user having mood information set toangry, processing circuitry 306 of the server displays a warning messageindicating that the first user is angry. The processing circuitry maycontinue to display these warnings until the mood of the first user ischanged to a mood that is calmer than angry.

In some embodiments, a ring tone corresponding to a first user may bemodified as a function of the mood of the first user, which may bedetermined based on mood information posted to a social network asdescribed above. For example, in response to receiving a request fromthe first user to access a primary content asset associated with anangry mood, processing circuitry of a primary user device may post firstmood information to a first social network account of the first user. Asecond user device of a second user, such as a smart phone, may modify afirst ringtone corresponding to the first user as a function of the moodinformation posted on the social network. For example, processingcircuitry of a second user device may access the first social networkaccount of the first user and retrieve the first mood information. Inresponse to retrieving the mood information, processing circuitry of thesecond user device may change a ringtone corresponding to the first userto indicate that the first user is angry. The ringtone may includeaudio, images, video or any other suitable content that may be played ordisplayed on a personal user device.

In some implementations, processing circuitry of the second user devicemay display an indicator or message next to a first user listing in anaddress book, indicating mood information for a first user associatedwith the first listing. The indicator may inform a second user of thesecond user device of the mood of the first user. For example,processing circuitry of the second user device may display a message“You're about to call your boss, Monty Gollum, but he's pissed . . . yousure you want to do that?” in response to retrieving mood informationfrom a social network account of a first user, Monty Gollum, indicatingthat Monty Gollum is angry. If the second user sees that the first useris angry, he may decide not to call the first user.

In some embodiments, mood information about a user may be used in thesearch and selection of relevant supplemental content asset, previouslydescribed in reference to step 850 of FIG. 8. For example, a user mayaccess an action video game on a gaming console located near a secondarystereo device. If the user is in an angry mood, the processing circuitryof a secondary stereo device may return songs related to a heavy metalcategory that is associated with anger and action. If the user is in acalm mood, processing circuitry of the stereo may return songs relatedto fast-paced classical music that is associated with action.

In some embodiments, mood information may be used to makerecommendations for content to be accessed. For example, processingcircuitry of a user device may determine a user's mood by retrieving themood information from a social network. Based on the mood information,the processing circuitry of the user device may display recommendedcontent with mood metadata corresponding to the user's mood information.

In some embodiments, mood information may be used to select asupplemental content asset that is relevant to a primary content asset.For example, processing circuitry 306 of a primary user device mayretrieve mood information about a user from a social network, andtransmit that information to a secondary device, for use in searchingfor relevant content assets by mood. For example, the secondary devicemay search for content assets with mood metadata information thatmatches the user's mood. In some embodiments, processing circuitry 306of a primary user device may determine mood information of a primarycontent asset accessed by a user and transmit that mood information to asecondary device for use in searching for relevant content assets bymood.

In some embodiments, mood information may be used to modify the viewingenvironment of a user. Systems and methods for customizing viewingenvironment preferences are discussed in greater detail in connectionwith Hays, U.S. patent application Ser. No. 13/172,533, filed Jun. 29,2011, which is hereby incorporated by reference herein in its entirety.For example, settings of viewing environment devices such as lights,window blinds, or any other suitable viewing environment device thatmodifies an aspect of the user's viewing environment may be modified asa function of the user's mood, or the mood of an accessed primarycontent asset. For example, if processing circuitry of a primary userdevice determines that a user is in a happy mood, or that a primarycontent asset accessed by a user corresponds to a happy mood, theprocessing circuitry of the primary user device may modify the settingsof viewing environment devices such as lights for maximum brightness tocreate a cheery environment. If a primary user device determines that auser is in a sad mood, or that a primary content asset accessed by auser corresponds to a sad mood, the primary user device may modify thesettings of viewing environment devices such as lights for minimumbrightness to create a gloomy environment.

In some embodiments, a secondary device may synchronize presentation ofa relevant supplemental content asset with the primary user device. Forexample, a stereo may start playing a movie soundtrack at a certainscene in a video game being accessed on the primary gaming console. Atstep 870, a secondary device sends a feedback information message to theprimary user device. The feedback information message may indicate thenumber and type of relevant content that has been found. The number ofrelevant content found may range from 0 to the number of contentavailable on the secondary devices. The feedback information message mayinclude a timestamp at which the secondary device may start presentationof the supplemental content asset. At a secondary device, processingcircuitry 306 may transmit a message on I/O path 302 acrosscommunications network 414 to the primary user device.

At step 880, processing circuitry of the secondary device polls todetermine whether a synchronization message containing synchronizationinformation has been received from the primary user device. Thesynchronization message may include a timestamp and timeline. Thetimestamp may indicate a time for the secondary device to startpresentation of supplemental content asset. If the secondary deviceprovides a playlist of supplemental content asset, the timeline mayinclude markers to indicate when different supplemental content assetsfrom the playlist should be played back. At the secondary device,processing circuitry 306 may receive the synchronization message fromI/O path 302. If the message has been received from the primary userdevice, the process will continue to step 890. If the message has notyet been received from the primary user device, the process will loop atstep 880.

At step 890, processing circuitry of the secondary device presents therelevant content to the user. For example, a stereo device may startplayback of a movie soundtrack as a supplemental content asset toaccompany a video game accessed on a primary user device. The messagereceived at step 880 may include information such as start time ofplayback, and a timeline for playback of the device. Processingcircuitry 306 of the secondary device may direct the speakers to playback the soundtrack. In some embodiments, playback information may beused in the presentation of supplemental content. As described above,playback information may indicate the status and elapsed playback timefor a primary content asset. To more harmoniously present thesupplemental content asset, processing circuitry 306 of the secondarydevice may adjust the start time for play back of the supplementalcontent asset by an offset comparable to the elapsed playback time ofthe playback information received in step 830 or 840, or an offsetcomparable to the timing information received as part of thesynchronization message in step 880.

FIG. 9 illustrates an example of a display screen that may be displayedto a user on a primary user device or secondary device that provides asupplemental content asset in accordance with some embodiments of thepresent disclosure. Display screen 900 may include a content displaywindow 910, a message prompt 920 and a configure menu button 930. Insome embodiments, the display screen of FIG. 9 may be shown to a user ona primary user device. For example, when a user is operating a gamingconsole or loading a video game on a gaming console, display 900 may beshown to a user on a display device connected to the gaming console. Thedisplay informs the user that secondary devices may be available toprovide a supplemental content asset (e.g., audio soundtracks, slideshows, advertisements, etc.) while the user is playing the video game.

Content display window 910 may display a primary content asset that isto be supplemented by a supplemental content asset from a secondarydevice. Content display window 910 may display a supplemental contentasset from a secondary device. When a user accesses content on a primaryuser device, a prompt may be displayed to the user indicating thatsecondary devices have been detected that are able to provide asupplemental content asset to the primary content asset accessed on theprimary user device. In some implementations, a prompt 920 may bedisplayed querying whether the user would like to enable the providingof a supplemental content asset. In other implementations, thesupplemental content asset may be enabled automatically. Configurebutton 930 may enable a user to modify settings of a secondary device,to configure how a supplemental content asset is provided and what kindof supplemental content asset is provided. Configure button 930 mayenable a user to modify settings of the primary user device to configurewhat information is transmitted in a content information message to asecondary device. For example, a user may modify a message template tospecify that only audio content should be provided as a supplementalcontent asset. Configure button may bring up the display screen 1000 ofFIG. 10.

In some embodiments, the display screen 900 of FIG. 9 may be displayedto a user by a secondary device. The display screen 900 may alert a userof a secondary device to other secondary devices that may provide asupplemental content asset, or to detected primary user devices that maybe supplemented by the secondary device. For example, a system ofsecondary devices may include a television which acts as a nexus, aset-top box, a television, and a stereo. The display 900 may be shown toa user on the television display to indicate that there are othersecondary devices (e.g., the set-top box and stereo) that may provide asupplemental content asset. The display 900 may be shown to a user toindicate that there may be one or more primary user devices (e.g. agaming console) for which a supplemental content asset may be providedby the secondary device or secondary devices. Media display window 910may show a content asset currently being accessed by the secondarydevice, the primary content asset accessed at the primary user device,or a supplemental content asset accessed from the secondary device thatmay be related to the primary content asset.

In response to receiving a request to access a content asset, a primaryuser device may transmit a content information message to a secondarydevice. At the secondary device, processing circuitry 306 may receivethe content information message from I/O path 302. In response toreceiving the content information message, processing circuitry 306 maydirect display 312 to show the display screen 900 to the user.Processing circuitry 306 may direct display 312 to display menu prompt920 to the user querying whether the user would like to enable thesecondary equipment to supplement the primary content asset accessed atthe primary user device. Processing circuitry 306 may direct display 312to display configuration button 930 that enables a user to configuresettings of the secondary device. Processing circuitry 306 may directdisplay 312 to show content window 910 that displays a primary contentasset or a supplemental content asset. The display of a supplementalcontent asset will be discussed in further detail in reference to FIG.13.

In some embodiments, a prompt message indicating that devices aredetected for providing a supplemental content asset may be displayed bya secondary nexus device in response to detecting other slave secondarydevices, or in response to detecting primary user devices. For example,secondary set-top box 510 may act as a secondary nexus device and detecta slave secondary stereo device 530 and a slave secondary televisionequipment 520. In response to detecting other slave secondary devices,secondary set-top box 510 may show screen 900 to a user on secondarytelevision equipment 520. At the secondary nexus device, processingcircuitry may receive information messages from other slave secondarydevices from I/O path 302, and in response transmit a command on I/Opath 302 across communications network 414 to slave secondary televisionequipment 520 to display screen 900 to the user. Slave secondarytelevision equipment 520 may receive the command on I/O path 302 fromcommunications network 414 and display screen 900 on display 312 ofslave secondary television equipment.

FIG. 10 illustrates an example of a display screen that may be shown toa user on a primary user device or secondary device to configure whatsupplemental content asset is provided and how the supplemental contentasset is provided in accordance with some embodiments of the presentdisclosure. For example, configuration screen 1000 may be shown to auser of a primary gaming console on a display device output from theprimary gaming console. Configuration screen 1000 may be shown to a userof a primary laptop computer.

In some embodiments, configuration screen 1000 of FIG. 10 may bedisplayed to a user in response to selection of configuration button 930from display screen FIG. 9. For example, configuration screen 1000 maybe shown to a user of a primary gaming console after accessing a videoby the gaming console in response to receiving a user input of theconfiguration button 930. Configuration screen 1000 may be shown to auser of a secondary nexus device, in response to receiving a user inputof configuration button 930.

At the secondary nexus device, in response to receiving a user inputfrom user input interface 310 indicating selection of configurationbutton 930, processing circuitry 306 may transmit a command to slavesecondary television equipment 520 to display the configuration screen1000 of FIG. 10. In some embodiments a single secondary device mayreceive the input and display the command screen. For example, atsecondary television equipment 520, processing circuitry 306 may receivean input from user input interface 310 indicating selection ofconfiguration button 930 from display screen 900 of FIG. 9. In response,processing circuitry 930 may show configuration screen 1000 on display312.

Configuration screen 1000 may display two menu items: a secondarydevice/correlation set up button 1010, and a supplemental content assetbutton 1020. In response to selection of button 1010, a user device maydisplay set-up screen 1100 of FIG. 11 to a user. In response toselection of button 1020, a user device may display selection screen1200 of FIG. 12 to a user. At a user device, processing circuitry 306may receive an input from user input interface 310 indicating selectionof a configuration button 1010, and in response output set-up screen1100 on display 312. At a user device, processing circuitry 306 mayreceive an input from user input interface 310 indicating selection of aconfiguration button 1020, and in response output set-up screen 1200 ondisplay 312.

FIG. 11 illustrates an example of a display screen 1100 that may beshown to a user on a primary user device or secondary device to set upthe providing of a supplemental content asset in accordance with someembodiments of the present disclosure. For example, a user may own aprimary gaming console, and a system of secondary devices including aset-top box which acts as a nexus, a stereo and a DVD player. Fromset-up screen 1100, a user may select secondary devices that provide asupplemental content asset (e.g. stereos, set-top boxes, laptopcomputers), type of a supplemental content asset (e.g. audio, video,advertisements, web content, etc), and how the supplemental contentasset is selected (e.g. metadata, feature-matching, profile matching).

In some embodiments, set-up screen 1100 allows a user to enable, disableor select specific secondary devices to be used to supplement content.Submenu 1110 may enumerate content sources that include a set-top boxmedia center, represented by menu item 1120, and a stereo systemrepresented by menu item 1132. The media center may be implemented in asecondary set-top box device 510. The stereo system may be implementedin a secondary stereo device 530.

A user may browse through a series of selectable options and drop-downmenus to navigate set-up screen 1100. As described in reference to step820 of FIG. 8, the media center may include a database of contentlistings that is represented by content library menu item 1124. A usermay navigate the secondary device submenu 1110 of screen 1100 byselecting the media center menu item 1120, and opening the contentlibrary menu item 1124. Within content library menu item 1124, a usermay select video menu item 1126, or music menu item 1128. In someembodiments, selecting the video menu item 1126 may bring up a listingof relevant video content, and selecting music menu item 1128 may bringup a listing of relevant music content. The media center menu item mayalso contain a web content menu item 1130. In some embodiments, themedia center may be able to access web pages, flash videos or otherInternet content that is relevant to the primary content asset accessedon the primary user device.

Under the stereo system menu item 1132, there may also be contentlibrary menu item 1134, different from the content library menu item1124 corresponding to the media center. The stereo system contentlibrary corresponds to content that is stored on the stereo system,unlike the media center content library which corresponds to contentstored or accessible by the media center. In some embodiments, thestereo system content library may be incorporated into the media centercontent library. For example, when a stereo system is a slave secondarydevice to a secondary nexus device, the secondary nexus device maygenerate a database or content library that includes content from allslave devices.

In some embodiments, set-up screen 1100 allows a user to select types ofcontent to provide as a supplement to a primary content asset accessedat the primary user device. For example, supplemental content asset menu1140 may include selectable menu items for audio type content 1142,video type content 1144, advertisements content 1146, and web content1148. Audio type content may include mp3 files, wav files, or anysuitable music or audio file. Video type content may include AVI files,MPEG files, 3gp files or any suitable video file. Web type content mayinclude websites, flash files, shockwave files, applets or any othersuitable Internet related content. Advertisements may include an audiocontent, video content or web content.

In some embodiments, set-up screen 1100 allows a user to select optionsfrom correlation type menu 1150 used to determine correlation andrelevancy of supplemental content asset items stored at a secondarydevice to a content asset accessed on a primary user device. Asdescribed above in reference to step 850 of FIG. 8, the relevancy ofsupplemental content asset items to a content asset accessed on aprimary user device may be determined using various information,including metadata of the content, features extracted from the content,and user profile information, as described above in reference to step820 of FIG. 8.

In some embodiments, a user may selectively enable or disable each ofthe menu items shown in set-up screen 1100. The user has flexibility toenable and disable various menu options. For example, a user may decideto enable the media center menu item 1120 and disable the stereo systemmenu item 1132. This limits the selection of a supplemental contentasset to the content assets provided by the media center. As an example,a user may limit the content types under supplemental content menu 1140to video content by enabling video menu item 1144, or to audio contentenabling audio menu item 1142.

FIG. 12 illustrates an example of a display screen 1200 that may beshown to a user on a primary user device or secondary device to selectsupplemental content asset in accordance with some embodiments of thepresent disclosure. For example, a user of a primary gaming console mayselect the type of supplemental content asset or specific supplementalcontent asset from selection screen 1200 to accompany a primary videogame accessed on the gaming console. The user may be playing an actionvideo game. The relevant supplemental content asset listings in displayscreen 1200 correspond to action themes or fast-paced music.

Display screen 1200 may be shown to a user by a user device in responseto receiving an input from a user indicating selection of supplementalcontent asset button 1020 in display screen 1000 of FIG. 10. Displayscreen 1200 shows a listing of relevant content items under severaldifferent content types. The relevant content items may have beendetermined as a result of a search for relevant content items as part ofstep 850 of FIG. 8. For example, the relevant content items listed indisplay screen 1200 may have been determined based on searching forcontent items relevant to a video game. Video options menu item 1210provides a listing of video content that is relevant to the accessedprimary content asset on the primary user device. In this example, menuitem 1212 refers to video clips.

Audio options menu item 1220 provides a listing of audio content that isrelevant to the accessed primary content asset on the primary userdevice. In this example, menu item 1222 refers to Tchaikovsky's “1812Overture,” and menu item 1224 refers to a playlist by Linkin Park. Insome embodiments, the menu items may refer to playlists such as menuitem 1224. In some embodiments the menu items may refer to individualcontent files. In some embodiments, playlists may be generated based onthe relevant content. For example, if more than one audio content islisted under audio options menu item 1220, the secondary device may beable to combine the several files into a playlist.

Web options menu item 1230 provides a listing of content that isrelevant to the primary content asset accessed on the primary userdevice. In this example, web options include a menu item referring tosoundtracks that may be streamed or downloaded from the Internet. Insome embodiments, users may purchase content that may be streamed ordownloaded. Menu item 1234 is a button that enables a user to purchasecontent from the Internet. In some embodiments, users may subscribe toshort-term licenses for audio files or video files that may be protectedby digital rights management (DRM). Users may periodically renewlicenses to this content. A menu item similar to the purchase menu item1234 may be displayed next to the content that requires a paidsubscription. For example, processing circuitry 306 of a user device mayreceive a request from a user to purchase a content asset. In responseto receiving the request, processing circuitry 306 may transmit apurchase request to a content source on the Internet. The purchaserequest may include metadata, user profile information, or any othersuitable information. In some implementations, the user device may be aprimary user device. In some implementations the user device may be asecondary device. In some implementations, the processing circuitry of aprimary user device may receive a purchase request from a user, andtransmit the purchase request to a second device. In response toreceiving the purchase request, processing circuitry of the secondarydevice may transmit information to a content source on the Internet.

In some embodiments each of the menu items in display screen 1200 may beindividually selected to enable or disable the supplemental contentasset. The ability to present multiple supplemental content assets to auser will be discussed in reference to FIG. 13.

FIG. 13 illustrates an example of a display screen that may be shown toa user on a secondary device displaying a supplemental content asset inaccordance with some embodiments of the present disclosure. For example,a primary user device may be a portable gaming device, accessing a songas a primary content asset. A secondary television equipment may displaya music video as a supplemental content asset in content window 1140, asaccompaniment to the playback of the song on the primary portable gamingdevice.

In some embodiments, display screen 1300 may be shown to a user on aprimary user device. For example, if the primary user device is atelevision equipment that also serves as a display for secondary set-topbox 510, the supplemental content asset may be displayed in a separatewindow or overlay, along with a primary video content. In someembodiments, display area 1340 may show a primary content asset accessedfrom a primary user device. For example, when a user is playing a videogame with primary gaming console 550, the video may be output tosecondary television equipment 520. In this example, a supplementalcontent asset in the form of advertisements and audio content may bepresented to the user through secondary television equipment 520,secondary set-top box 510, and secondary stereo device 530. Audiocontent may be played back by secondary stereo device 530 or secondarytelevision equipment 520 through speakers 540.

In some embodiments, multiple supplemental content asset may bepresented to the user in display screen 1300. For example, displayregion 1320 may show an advertisement accessed from the Internet, orfrom remote content source 416. Display region 1310 may indicate thatthe secondary device is providing a supplemental content asset. Time bar1330 may be presented to the user to assist in synchronizing thepresentation of a supplemental content asset and the accessed primarycontent asset. For example, relevant supplemental content asset may be aplaylist consisting of several audio content files. Markers 1332, 1334,1336 and 1338 may denote transition points between the parts of theplaylist. In some embodiments, the timebar may represent notabletransition points in the primary content asset accessed by the primaryuser device. In these embodiments, if the relevant content is aplaylist, the markers may indicate points at which each audio contentfile may start or finish. In some embodiments, the duration betweentransition points may be shorter than the audio content in a playlist.In these embodiments, the transition between audio content may besmoothed by a fade-in and fade-out of volume.

In some embodiments, supplemental video content may be displayed indisplay region 1140 instead of a primary content asset accessed at theprimary user device. For example, a user may wish to play a music fileon a primary PDA device 570. In response to receiving contentinformation messages about the content, secondary set-top box 510 mayselect video clips to supplement the audio content accessed by theprimary user device.

FIG. 14 illustrates an example of a display screen that may be shown toa user on a secondary device displaying a supplemental content asset inaccordance with some embodiments of the present disclosure. For example,a user may play a game on a primary user device such as a tabletpersonal computer (PC). A secondary device such as television equipment520 may provide a collection of a supplemental content asset to enhancethe user experience of playing the game.

In some embodiments, display screen 1400 may be shown to a user on asecondary device, such as a television, in response to accessing a videogame on a primary tablet PC. For example, a tablet PC may enable a userto play a car racing game. The amount of information that may bedisplayed to a user may be limited by the screen size of the tablet PC.Processing circuitry 306 of the primary tablet PC may communicate withsecondary set-top box 510 to access supplemental web content from theInternet that is relevant to the game. The primary tablet PC may be usedby a user to steer and drive along a course in the racing game. Asupplemental content asset may be displayed on secondary televisionequipment 520 by secondary set-top box 510. In some embodiments, thetablet PC may enable the user to compete with other players over theInternet. For example, processing circuitry 306 of the tablet PC mayreceive input from a user while playing the game and communicateinformation over a communications network with other tablet PCs, toexchange information about the status of different competing players ateach tablet PC. The display 1400 may include a display of an overheadmap of the racing course 1410, rankings of players and fastest times1420, a listing of friends currently playing the racing game 1430, titleof the racing game 1440, and an advertisement 1450. Selectable region1460 may include selectable menu options that enable a user to accessany of the configurations and set-up screens of FIG. 10, FIG. 11 andFIG. 12.

In some embodiments, audio may be provided as a supplemental contentasset. A system of secondary devices that include at least a set-top boxand stereo can provide enhanced sound effects. The set-top box mayaccess the sound effects from a remote media server that is communicatedfrom the primary user device that contains sound effects specific to theracing game. The set-top box may then play back the sound effects incoordination with game play on the tablet PC.

In some embodiments, any of display screens 900, 1000, 1100, 1200, 1300or 1400, as show in FIG. 9, 10, 11, 12, 13, or 14, respectively, may becustomized depending on the mood of the user of the primary user device,or the mood of the primary content asset. The customization may beimplemented as different skins that modify the graphical user interface(GUI) and appearance of the display screen. For example, in response todetermining that a user is in a sad mood, or that the primary contentasset corresponds to a sad mood, processing circuitry 306 of a userdevice may direct display 212 to show dark color schemes for buttons andborders. As described above, processing circuitry 306 of a user devicemay determine mood information based on: user profile information storedon the user device or retrieved from a remote source, user profile ormood information retrieved from an online social network, or the primarycontent asset accessed by the user.

FIG. 15 illustrates an example of a data structure of an informationmessage 1500 for transmitting information between a primary user deviceand a secondary device in accordance with some embodiments of thedisclosure. For example, the information may be communicated about aprimary content asset accessed on a primary user device for determininghow and what supplemental content asset is provided by a secondarydevice. The data structure may include metadata 1510, user profileinformation 1530, supplement request information 1540, featureinformation 1550, synchronization information 1560, capabilitiesinformation 1570, and playback information 1580. Metadata 1510 mayinclude information from a header 1520 of the content such as title,genre, content type or any other suitable metadata information. Userprofile information 1530 may include user name, gender, age, race,preference information, or any other suitable user profile information.

In some embodiments, the data structure may also include a SupplementRequest field 1540 that denotes the content type that is preferred forthe supplemental content asset. For example, if a user is playing a songon a primary PDA device 570, the user may request video content tosupplement the song. As an example, if a user is playing a video game ona primary gaming console 540, the type of a supplemental content assetpreferred may be a soundtrack played back on secondary stereo equipment530.

In some embodiments, feature information 1550 may be included with theinformation message. Acoustic features and visual features informationmay be extracted from the accessed primary content asset. For example,if the accessed primary content asset is a song, MFCCs may be extractedfrom a Fourier spectrum of the song. These MFCCs provide informationabout the frequency band components of the song. The beat or cadence ofthe song may be extracted by quantifying low frequency components of aFourier spectrum of the song. This may be used to match the song torelevant music videos or other audio files that have similarcharacteristics. In FIG. 15, feature information 1550 includes amagnitude measurement of a bass component.

In some embodiments, synchronization information 1560 may be included inan information message. Synchronization information messages andfeedback information messages were previously described in connectionwith FIG. 7 and FIG. 8. The structure of those messages may be similarto message 1500. The synchronization information may include a timestamp1561 indicating when a primary or a supplemental content asset should bepresented. For example, timestamp 1561 indicates that a supplementalcontent asset should be played at one minute and 10 seconds after theprimary content asset is played back. The synchronization informationmay include a timeline with markers that indicate when a supplementalcontent asset should be presented to supplement a primary content asset.The timeline may be communicated in the message as an array of offsets1562 from timestamp 1561, and an array of content indices 1563 thatcorrelate with a supplemental content asset. For example, array 1563indicates that a supplemental content asset corresponding to index 1should be played back at timestamp 1561, a supplemental content assetcorresponding to index 2 should be played back at five minutes aftertimestamp 1561, and a supplemental content asset corresponding to index3 should be played back at fifteen minutes after timestamp 1561. In someimplementations, the same audio clip may be played at each marker, or aseries of different audio clips in a playlist may be played at eachmarker.

In some embodiments, capabilities information 1570 may be included in aninformation message, such a device information as discussed previouslyin reference to FIG. 7. As described previously in reference to step 720of FIG. 7, the capabilities field 1570 may be included in a deviceinformation message to communicate the abilities of a secondary device.As described in reference to step 730 of FIG. 7, processing circuitry306 of a user device may create a message template based on thecapabilities information received from a secondary device. Capabilitiesinformation may include information about the types of content a userdevice is able to present. For example, the capabilities informationcorresponding to a stereo device may indicate that the device supportsaudio only. The capabilities information corresponding to a laptopcomputer may indicate that the laptop supports audio and video. Forexample in FIG. 15, capabilities information includes supported contenttags 1571 and 1572. Supported content tag 1571 indicates that a userdevice supports audio content, and supported content tag 1572 indicatesthat the user device supports video content.

In some embodiments, playback information 1580 may be included in aninformation message, such as a content information message, proberesponse message, probe message, beacon message, feedback informationmessage or synchronization information message. The status field 1580indicates the status of a content asset on a user device and may be anyone of paused or playing back. Paused indicates that the content asseton a user device transmitting the playback information is currentlypaused. Playing back indicates that the content asset on the user deviceis currently playing. Play time field 1582 indicates the elapsedplayback time of the content asset. Total time field 1583 indicates thetotal time duration for playback of the content asset. The playbackinformation may be used to synchronize presentation of supplementalcontent assets and primary content assets, in particular when primarycontent assets have been partially played back. For example, if theprimary content asset has been played half way, the secondary device maystart playback of the supplemental content asset at halfway through thetotal time duration of the supplemental content asset.

The above described embodiments of the present disclosure are presentedfor purposes of illustration and not of limitation, and the presentdisclosure is limited only by the claims which follow.

What is claimed is:
 1. A method comprising: detecting a primary userdevice within a proximity of a secondary user device; receiving, fromthe primary user device, an electronic communication at the secondaryuser device, wherein the electronic communication includes playbackinformation about a primary content asset being accessed at the primaryuser device; in response to receiving the electronic communication andwhile the primary user device is within the proximity of the secondaryuser device, automatically searching, using the received playbackinformation, a content listings database to identify a content listingthat is relevant to the primary content asset, wherein the searchingcomprises: automatically computing, for each content listing in thedatabase, a relevancy metric for a supplemental content asset associatedwith the respective content listing based on a characteristic of theprimary content asset, and selecting, as the identified content listing,the content listing associated with the supplemental content asset forwhich the relevancy metric exceeds a threshold; retrieving thesupplemental content asset associated with the identified contentlisting in response to selecting the identified content listing; andgenerating for display at the secondary user device, the retrievedsupplemental content asset that is relevant to the primary content assetbeing accessed at the primary user device, based in part on the receivedplayback information.
 2. The method of claim 1, further comprisinggenerating the content listings database, wherein the content listingsdatabase includes at least one content listing associated with asupplemental content asset accessible to the secondary user device. 3.The method of claim 2, wherein the secondary user device is a nexusdevice, further comprising: detecting a slave secondary device within aproximity of the nexus device; receiving, from the slave secondarydevice, a second electronic communication, wherein the second electroniccommunication includes content listing information about at least onesupplemental content asset accessible to the slave secondary device;updating the content listings database, based on the received secondelectronic communication, to include the content listing informationabout the supplemental content asset accessible to the slave secondarydevice.
 4. The method of claim 1, wherein the characteristic of theprimary content asset comprises metadata, timing information, userprofile information, mood information, play back information, or featureinformation.
 5. The method of claim 4, wherein the feature informationof the primary content asset or content listing further comprises afeature vector; wherein the computing, for each content listing in thedatabase, the relevancy metric for the supplemental content assetassociated with the respective content listing further comprises:preprocessing a feature vector of the primary content asset.
 6. Themethod of claim 5, wherein the preprocessing further comprises resizinga feature vector, combining elements of a feature vector, or normalizinga feature vector; and wherein the computing the relevancy metric furthercomprises calculating a result vector.
 7. The method of claim 4 whereinthe feature information comprises a mel-frequency cepstral coefficient,cepstral coefficient, formant, frequency, brightness, contrast, colorrange, edge count, or corner count.
 8. The method of claim 1 furthercomprising: generating, using a processor, a feedback informationmessage that includes metadata, or timing information about theidentified content listing; and transmitting the feedback informationmessage to the primary user device.
 9. The method of claim 1 furthercomprising: receiving, from the primary user device, a synchronizationinformation message, wherein the synchronization information messageincludes timing information; and the presenting of the retrievedsupplemental content asset is in response to receiving thesynchronization information message and based on the timing informationincluded in the synchronization information message.
 10. The method ofclaim 1, further comprising: receiving from the primary user device arequest to purchase a content asset from a content source over theInternet.
 11. A secondary user device comprising: storage; and controlcircuitry configured to: detect a primary user device within a proximityof the secondary user device; receive, from an I/O path, an electroniccommunication from the primary user device, wherein the electroniccommunication includes playback information about a primary contentasset being accessed at the primary user device; in response toreceiving the electronic communication and while the primary user deviceis within the proximity of the secondary user device, automaticallysearch, using the received playback information, a content listingsdatabase to identify a content listing that is relevant to the primarycontent asset, by: automatically computing, for each content listing inthe database, a relevancy metric for a supplemental content assetassociated with the respective content listing based on a characteristicof the primary content asset, and selecting, as the identified contentlisting, the content listing associated with the supplemental contentasset for which the relevancy metric exceeds a threshold; retrieve thesupplemental content asset associated with the identified contentlisting in response to selecting the identified content listing; andgenerate for display at the secondary user device, the retrievedsupplemental content asset that is relevant to the primary content assetbeing accessed at the primary user device, based in part on the receivedplayback information.
 12. The secondary user device of claim 11, whereinthe control circuitry is further configured to: generate the contentlistings database, wherein the content listings database includes atleast one content listing associated with a supplemental content assetaccessible to the secondary user device.
 13. The secondary user deviceof claim 12, wherein the secondary user device is a nexus device, andthe control circuitry is further configured to: detect a slave secondarydevice within a proximity of the nexus device; receive, from the slavesecondary device, a second electronic communication, wherein the secondelectronic communication includes content listing information about atleast one supplemental content asset accessible to the slave secondarydevice; update the content listings database, based on the receivedsecond electronic communication, to include the content listinginformation about the supplemental content asset accessible to the slavesecondary device.
 14. The secondary user device of claim 11, wherein thecharacteristic of the primary content asset comprises metadata, timinginformation, user profile information, mood information, play backinformation or feature information.
 15. The secondary user device ofclaim 14, wherein the feature information of the primary content assetor content listing further comprises a feature vector; wherein thecontrol circuitry is further configured to: preprocess a feature vectorof the primary content asset.
 16. The secondary user device of claim 15,wherein the control circuitry is further configured to: resize a featurevector, combine elements of a feature vector, or normalize a featurevector; and calculate a result vector to compute the relevancy metric.17. The secondary user device of claim 14 wherein the featureinformation comprises a mel-frequency cepstral coefficient, cepstralcoefficient, formant, frequency, brightness, contrast, color range, edgecount, or corner count.
 18. The secondary user device of claim 11,wherein the control circuitry is further configured to: generate, usinga processor, a feedback information message that includes metadata, ortiming information about the identified content listing; and transmitthe feedback information message to the primary user device.
 19. Thesecondary user device of claim 11, wherein the control circuitry isfurther configured to: receive, from the primary user device, asynchronization information message, wherein the synchronizationinformation message includes timing information and the presenting theretrieved supplemental content is in response to receiving thesynchronization information message and based on the timing informationincluded in the synchronization information message.
 20. The secondaryuser device of claim 11, wherein the control circuitry is furtherconfigured to: receive, from the primary user device, a request topurchase a content asset from a content source over the Internet.
 21. Anon-transitory computer-readable media, the computer-readable mediacomprising machine-readable instructions encoded thereon for: detectinga primary user device within a proximity of a secondary user device;receiving, from the primary user device, an electronic communication atthe secondary user device, wherein the electronic communication includesplayback information about a primary content asset being accessed at theprimary user device; in response to receiving the electroniccommunication and while the primary user device is within the proximityof the secondary user device, automatically searching, using thereceived playback information, a content listings database to identify acontent listing that is relevant to the primary content asset, whereinthe searching comprises: automatically computing, for each contentlisting in the database, a relevancy metric for a supplemental contentasset associated with the respective content listing based on acharacteristic of the primary content asset, and selecting, as theidentified content listing, the content listing associated with thesupplemental content asset for which the relevancy metric exceeds athreshold; retrieving the supplemental content asset associated with theidentified content listing in response to selecting the identifiedcontent listing; and generating for display at the secondary userdevice, the retrieved supplemental content asset that is relevant to theprimary content asset being accessed at the primary user device, basedin part on the received playback information.
 22. A method comprising:detecting a primary user device within a proximity of a secondary userdevice; receiving, from the primary user device, an electroniccommunication at the secondary user device, wherein the electroniccommunication includes playback information about a primary contentasset being accessed at the primary user device; in response toreceiving the electronic communication and while the primary user deviceis within the proximity of the secondary user device, automaticallysearching, using the received playback information, for a supplementalcontent asset associated with the primary content asset; automaticallycomputing a relevancy metric for the supplemental content assetassociated with a content listing in a database based on acharacteristic of the primary content asset; and generating for displayat the secondary user device, the retrieved supplemental content assetthat is relevant, based on the computed relevancy metric, to the primarycontent asset being accessed at the primary user device, based in parton the received playback information.
 23. A secondary user devicecomprising: storage; and control circuitry configured to: detect aprimary user device within a proximity of the secondary user device;receive, from an I/O path, an electronic communication from the primaryuser device, wherein the electronic communication includes playbackinformation about a primary content asset being accessed at the primaryuser device; in response to receiving the electronic communication andwhile the primary user device is within the proximity of the secondaryuser device, automatically search, using the received playbackinformation, for a supplemental content asset associated with theprimary content asset; automatically compute a relevancy metric for thesupplemental content asset associated with a content listing in adatabase based on a characteristic of the primary content asset; andgenerate for display at the secondary user device, the retrievedsupplemental content asset that is relevant, based on the computedrelevancy metric, to the primary content asset being accessed at theprimary user device, based in part on the received playback information.24. A non-transitory computer-readable media, the computer-readablemedia comprising machine-readable instructions encoded thereon for:detecting a primary user device within a proximity of a secondary userdevice; receiving, from the primary user device, an electroniccommunication at the secondary user device, wherein the electroniccommunication includes playback information about a primary contentasset being accessed at the primary user device; in response toreceiving the electronic communication and while the primary user deviceis within the proximity of the secondary user device, automaticallysearching, using the received playback information, for a supplementalcontent asset associated with the primary content asset; automaticallycomputing a relevancy metric for the supplemental content assetassociated with a content listing in a database based on acharacteristic of the primary content asset; and generating for displayat the secondary user device, the retrieved supplemental content assetthat is relevant, based on the computed relevancy metric, to the primarycontent asset being accessed at the primary user device, based in parton the received playback information.